How to analyse aid: using statistical analysis to improve effectiveness

There is probably no other question in economics that evokes such strong emotions and creates such a clear divide between left and right. To the left, the world’s poor are caught in a poverty trap; without the minimum resources to help them help themselves, they will never break out of this cycle. Aid, then, is clearly needed as a big push to kick things off. To the right, aid encourages dependency, distorts markets and props up nasty regimes – keeping the poor in their place. There is no sign of either side winning this battle. But subjecting individual aid projects to robust statistical analysis through randomised controlled trials (RCTs), modelled on medical trials, could provide a way out of the ideological mire.

So what is aid? Aid can come in many forms – such as military aid to help a country fight terrorists, or budgetary aid sent with no strings attached to bolster a government’s finances, or the cancelling of historic debts. Here we focus on a more altruistic form of aid – that sent to help the world’s poorest with little expectation of benefit to the donor, for example direct financing of mosquito nets or primary schools.

The moral cause for wishing to alleviate world poverty is a clear one. That one billion people live on this planet on less than a dollar a day while so many of us live in luxury is surely a moral outrage. The philosopher Peter Singer has compared such inaction to standing by a lake as a child drowns. Surely, he reasons, no person would give a seconds thought to diving in? Practically speaking, of course, giving aid is complex and costly yet the principle, he argues, is the same. This metaphor succinctly encapsulates the argument for aid – that only immediate, decisive and potentially painful action, essentially diving in, can help the poor. The clear implication of Singer’s reasoning is that the solution is to get actively involved – but what if modern aid offers our metaphorical child little more than a faulty life ring? Or worse, what if aid actually helps to drag her down? This is the real question in the aid debate. This goes beyond rock concerts and emotive advertisements; is the obvious solution the right one?

Diving In Many would say yes. These aren’t solely rock stars and actors – distinguished professors of economics such as Columbia’s Jeffery Sachs see aid as an essential way of unlocking the economic potential of the poor. In his book The End of Poverty, Sachs points to farming as an example of this – if farmers are donated fertiliser, they can massively increase their harvest, which in turn gives them money to be able to buy their own fertiliser next year, thus creating a virtuous cycle. For Sachs, the farmers are caught in a poverty trap (unable to buy the fertiliser they need to progress) and without an aid-based kick-start (free fertiliser) they will never escape from poverty. Sachs sees this poverty trap model applying to many other areas of development – sick, poor people cannot afford medicine, but whilst they are ill they are unable work, meaning that they get even poorer. But aid also has a larger role to play. It can help small, cash strapped democracies stay on their feet and provide for their people’s basic needs. It can also kick-start basic infrastructure such as roads and schools. Sufficiently compelled by this call to action, I headed off with some other Imperial College students last year to witness just this type of aid – designed to kick-start economic progress – in rural Kenya. Herdsmen, caught by a severe drought, were encouraged to take up fishing. A British charity subsidised nets and boats for the fledgling fishermen, with the idea that the subsidies could be gradually reduced as the economy took off. Each boat would take ten fishermen and each fisherman supported around ten dependents, meaning 100 people were helped by just one boat. It seemed to be the perfect project.

Unforeseen Consequences There are, however, powerful arguments against this kind of help. On a nationwide scale, the large and sudden influx of foreign money makes the local currency rise in value, suffocating exports. This was best documented in Holland when the manufacturing sector contracted, following the discovery of natural gas in 1959, earning the phenomenon the name Dutch disease. Furthermore, money given to poor country governments needn’t necessarily end up going to infrastructure or healthcare. According to Paul Collier of Oxford University, 40% of African military spending is funded by aid, unbeknownst to the donors. It is certainly no secret that the poorest countries often have the nastiest regimes. Aid can also undermine democracy, making poor countries accountable to donors, not to their citizens. The old American maxim of ‘no taxation without representation’ too often works in reverse – no representation without taxation. When poor countries rely on foreign funds and not taxes from local people and businesses, the need to be accountable to citizens is reduced and corruption and inefficiencies can more easily set in. Somaliland provides an excellent counter-example – not recognised as a country by the international community, it cannot receive aid and in fact has seen significant improvements in accountability and reduced corruption as citizens demand more for their taxes. And lastly, we cannot forget the ethical implications of aid recipients relying on public services provided by politicians in wealthy countries for whom they cannot vote.

The Truth Lies in the Data Where does this leave the Kenyan fishermen? They lack a mature manufacturing sector so that shouldn’t be an issue. But how could the other factors affect them? It is here that we get to one of the biggest problems with aid – the lack of good data on its effectiveness. In truth neither the left nor right perspectives can offer a good evaluation of such a program without first gathering data. This was exactly what we did in Kenya.

Through interviewing 200 of the fishermen, we were able to gain a clearer picture of how the subsidies were impacting the local economy. The results were surprising; the perfect picture offered by aid evangelists was found to be murky at best. Investment in boats turned out to be not only a surprisingly ineffective way of catching fish – as opposed to just tying nets to rocks and leaving them – but was also heavily biased in favour of the relatively rich fishermen who could afford to pay the unsubsidised half of the price. However, we also found that investment in nets, especially targeted at the poorest, was a very cost effective way of helping out, with nets paying for themselves after two months. This was not only interesting, it was also very useful. The data enabled the charity to refocus its efforts, saving valuable donations and improving the quality of life for the fishermen. Our research is part of a bigger shift in the aid debate – its scientific revolution. In seeking out quality data, we were able to get to the heart of the issue. The apex of this is the use of randomised controlled trials (RCTs), in the style of medical trials, on aid projects. The idea is simple. Half of a population is given a form of aid (nets, extra teachers etc.) while the other isn’t. Those who do and don’t receive the aid are chosen at random and in such a way that they cannot affect each other so that the impact of the aid can be clearly demonstrated. Essentially like any lab experiment. The idea is simple and common sense to any science student but it is having a significant impact on a hitherto opaque world.

The world before evidence-based medicine is hard to imagine – quack doctors of varying credibility combined folklore and pseudoscience to propose cures for just about any illness. Today, trials form the basis of medicine. Think of the 1991 discovery that folic acid reduces incidence of spina bifida, as a result of which 10% of the world’s flour is now fortified with the acid and thousands of such birth defects have been prevented. This same process is gradually taking place in aid. RCTs in Uganda have found that providing food in schools won’t increase enrolment, but it does boost attendance. RCTs across Africa have found that charging even a nominal fee for a malaria bed greatly reduces their use, contrary to the intuitive view that people only value what they pay for. Aid is experiencing its own scientific revolution.

The pioneers of this field are undoubtedly Professors Abhijit Banerjee and Esther Duflo of MIT’s Poverty Action Lab, who have been working in this area since the Lab’s founding in 2003. In their 2011 book Poor Economics, they outline the results of their RCT work. Their findings agree with neither left nor right, they simply find out what works. A new student union project at Imperial College called The African Development Project is dedicated to doing just this; involving science and engineering students in the evaluation of aid projects.

The potential of RCTs is very exciting indeed. They can provide no magic bullet – far too many such cures have been proposed in the past – but they do provide a way to go beyond the ideological warfare and change the aid debate for the better. Beyond simple moralising and abstract philosophising, scientific trials offer perhaps the most effective approach yet to understanding and tackling poverty. Its message is simple: find out what works, and then fund what works.

Imperial students interested in learning more about what makes an effective charity might want to attend “How To Run The Most Effective Charity In The World” on Thursday 5th December, 6pm Huxely 145, a panel discussion with the directors of two of GiveWell’s 3 “most effective charities in the world 2013” and the director of a charity consultancy. Email afrodev@ic.ac.uk for info.