Lesson in poverty eradication humility

Micheal Clemens has a nice post that seeks to make out a case for long-term evaluation (comparison of the villages that get the intervention to villages that do not get it, in such a way that which villages do and don’t get the intervention are randomly picked from an initial group) of development interventions since short-term effects and long-term effects can be completely different from one another.

But he points to a massive experiment, the World Bank financed Southwest Poverty Reduction Project in China, a multi-sectoral village-level development package intervention in 1,800 rural villages, whose failure is an object lesson in bringing some humility to poverty eradication experts.

It was implemented in the autonomous region of Guangxi and the provinces of Guizhou and Yunnan, and covered 600,000 households in 35 of China’s poorest counties, and ultimately assisted 2.8 million people. The World Bank provided $247.5 million for the project, and Beijing matched that amount.

The Project targeted the poorest villages, lasted about five years (1995-2000), cost hundreds of thousands of dollars per village, and sought to permanently reverse the fortunes of those villages with a broad-based and community participation-based package including roads, piped water, power lines, upgrading schools and clinics, training of teachers and health-care workers, microcredit, and initiatives for raising crop yields, animal husbandry, and horticulture.

Evaluations of the Project five years after it started, by Shaohua Chen, Ren Mu, and Martin Ravallion (also here), showed that income in those villages grew by 20% more during the project than in similar villages in the same area that had not received the intervention, and savings grew by 100% more. However, once the the intervention ended, all those effects on income and savings disappeared. Stunningly, ten years after the five-year project began, average income and savings in the villages that got that massive package of interventions were indistinguishable from income and savings in villages that did not.

A double-difference estimator of the program's impact (on top of pre-existing governmental programs) reveals sizeable short-term income gains that were mostly saved. Only modest gains to mean consumption emerged in the longer term - in rough accord with the gain to permanent income. Certain types of households gained more than others. The educated poor were under-covered by the community-based selection process - greatly reducing overall impact.

From hindsight and subsequent analysis, incomes in both the treated and untreated Chinese villages in the Southwest Project area increased greatly during the span of the project (1995-2000) and for years thereafter since the period coincided with the dramatic transformation of the Chinese economy. The impact, if any, of the massive village-level package development interventions, was marginal.

Btw, about the Clemens post itself, I could not but agree with Chris Blattman on RCTs, albeit in the context of a different project - the equally ambitious Millennium Villages Project.