Giving Up the Big Question

For most of the second half of the twentieth century, development economics argued about whether aid works.

The method was cross-country regression: line up dozens of countries, measure aid received and growth achieved, control for whatever seemed relevant, and look for a relationship. The results were contradictory for forty years. Some found aid raised growth, some found it lowered growth, some found it worked only with good policy, and each new paper was contested by the next.

The reason is that too many things vary at once. Countries receiving large aid flows differ from countries receiving small ones in every respect that matters: conflict, geography, colonial history, political structure, why they needed the aid in the first place. No set of controls fixes this, because the question as posed is not answerable with the data available.

Abhijit Banerjee, Esther Duflo and Michael Kremer's response was to stop asking it.

Instead of "does aid work", ask: does deworming raise school attendance? Do free bednets get used? Does hiring an extra contract teacher improve learning? Does microcredit lift incomes? Each of these can be tested with a randomised trial: assign the intervention to some villages or schools and not others, compare outcomes, and the comparison is credible because assignment was random.

The approach won a Nobel in 2019 and changed what development economists do. It has also been criticised sharply, and both the achievement and the criticism are worth taking seriously.

Start with what the trials found, because some results overturned confident assumptions.

The bednet question is the cleanest example. Malaria nets save lives; the question was whether to give them away or sell them cheaply. The argument for charging was familiar and plausible: people do not value what they get free, a price screens for those who will actually use the net, and cost recovery makes the programme sustainable.

Jessica Cohen and Pascaline Dupas tested it. Demand collapsed at small positive prices. Uptake fell by around three quarters when a modest charge was introduced. And usage rates among those who received nets free were essentially the same as among those who paid, so the screening benefit did not exist.

A plausible theory, tested, and settled. That is the case for the method in one experiment.

The second finding that changed practice concerns cash. The standard objection to giving poor people money is that it will be spent badly, and it is stated with great confidence by people who have never checked.

A review of nineteen studies found no systematic increase in spending on alcohol or tobacco, and reductions in several. Unconditional cash transfer trials in Kenya have found increases in assets, earnings, food security and psychological wellbeing persisting for years. GiveDirectly's programme was built on this evidence.

The consequence for the field is that cash has become the benchmark. The question asked of any new programme is no longer whether it produces benefits but whether it produces more benefits than simply giving the recipients the money it costs to run. A surprising number of well-regarded programmes do not clear that bar.

Now the criticisms, which are substantial.

The first is external validity. A trial establishes very cleanly what happened in those villages, with that implementation, at that time. Whether it transfers to a different country, a different scale, or a government rather than an NGO delivering it, is a separate question the trial does not answer. Angus Deaton and Nancy Cartwright have pressed hard on this: randomisation gives you internal validity, and internal validity is not what a policymaker in another country needs.

The field's response has been replication across multiple sites and more attention to mechanism, because understanding why something worked is what tells you where else it will work. An effect size travels badly; a mechanism travels better.

The second criticism is more fundamental. You cannot randomise a country into having an independent judiciary, an industrial policy, a functioning tax administration or a stable currency. The things that most plausibly explain why South Korea is rich and Ghana is not are exactly the things the method cannot study.

Lant Pritchett's version of this is that the movement has traded relevance for rigour, becoming excellent at evaluating interventions that are, on any reckoning of what drives national income, marginal. Growth of the kind that transformed East Asia does not come from better deworming.

The reply is that a small question answered is worth more than a large question argued about for forty years, and that the alternative on offer was the cross-country regression literature, which produced no reliable knowledge at all.

Both sides are right, which is why the field now contains both, somewhat uneasily.

One practical lesson from the scaling failures is worth carrying, because it generalises well beyond development. Programmes that work in a trial frequently fail when rolled out, and the reasons are predictable: the pilot was run by unusually motivated people, the supply of the necessary skilled staff runs out at scale, or general equilibrium effects appear that a small trial cannot detect (train everyone in a skill and the wage premium for it disappears).

Knowing the mechanism is what lets you predict which of these will bite. Which is the argument for replication before rollout, and against treating a single impressive effect size as a decision.