The enduring fallacies of development

Some of the enduring narratives of development, a frequent recent topic of my blogs, goes something like this,

A lot of development programs just fail because they’re trying to solve a problem that doesn’t exist. They’re just solving the wrong problem. The first really important thing you’ve got to do is really understand what the issue is in any given area. If we’re worried about girls not going to school because of menstruation, well, let’s start by finding out whether they actually don’t go to school more when they’re menstruating. That’s a really basic, obvious thing. But we actually need more work on that kind of understanding the context, understanding the problem, is really important first step.

Really??

Some one should engage on an unpacking exercise of this program to address girl children dropping out. Why is this program's design flawed? How does it not realise the objective? What are the alternatives? How implementable - fiscally supportable, administratively doable, and politically acceptable - are those alternatives? 

Sample this.

I did a project looking at how to improve immunization rates in India, which was fantastically effective. It started with a first assessment of what are the health problems in this area? Only 3% of kids in this part of India were getting fully immunized... There were a number of theories about why that could be... people... don’t trust the formal health system. There was also a question of, so the clinics are often closed, so is that the problem? Is it that when you go and take your kid to the clinic, it’s often closed? Is it nurse absenteeism that’s the problem? Or is it just a behavioral economics thing that you’re happy to get your kid immunized, but you’ll do it tomorrow? We read all this behavioral economics and we said, “Well, maybe we should look at that.” But we also wanted to test these other ideas. One arm made sure that without fail, there was someone to immunize your child and another arm did that, but also provide a small incentive. So, yes, we were testing a program but we were also asking a more fundamental question, which is, why don’t people get their kids immunized?

What we saw in the data is a lot of people got their kid immunized with one immunization, but they failed to persist to the end of the schedule. Which already, that’s just descriptive data and it starts to tell you, it’s not that they distrust the system or that they think that immunizations are evil, because they’re getting their kid one immunization. It’s more question of persistence. Now, fixing the supply problem increased the number of people getting the first shot, and the second shot, but again, it failed to fix this persistence problem. Where the incentive effect worked, was it helped people persist to the end. That tells you that one of the big problems was this persistence problem. It tells you a lot about why immunization isn’t happening. 

Blimey! What can a policy maker learn from this academically famous experiment on lentils? In any population group, people are likely to not end up immunising their children for a variety of reasons and these reasons would likely vary considerably across contexts. And in every context, the more perceptive among the nurses, and there are several of them, would invariably know exactly the primary reasons for the state of affairs with a fair degree of accuracy. It is a different matter that their concerns are rarely enquired and even more rarely acted upon. 

What was the need to conduct an expensive RCT with limited external validity to ascertain this? Why should we elevate the form of academic rigour associated with an experiment conducted by outsiders with little idea of the context over the substance of information discovery (by whatever means) from those living in those contexts and working on the same issue for years, even decades? Talk about arrogance of experts. 

If this story is narrated to an audience of practitioners engaged with the problem, it would only be out of courtesy that the narrator would escape being heckled.

This is some claim,

And actually, if you look at most of the people doing RCTs, they don’t think that their audience is aid donors. Their audience is the government of India and the government of Brazil and the government of Indonesia, and to some extent big companies or other individuals there.

Government of India and RCTs?? Didn't the last Chief Economic Advisor to the Government of India say this?

When asked how many of these expensive RCTs had moved the policy needle in India, Arvind Subramanian, Chief Economic Advisor, GOI, was hard pressed to find a single one that had been helpful to him in addressing the dozens of pressing policy questions that came across his table.

This claim about the policy relevance to the Government of India of RCTs in illuminating the big questions in education and resolving the most problems is just that, a researcher's self-serving claim. 

A lot of the work on education has suggested that the most effective thing we can do in education is to focus on the learning within the classroom. It’s not about more money, it’s not about more textbooks, it’s not about… And that’s what governments spend their money on. They spend it on teachers and textbooks, mainly teachers. But more teachers doesn’t actually improve learning. More textbooks doesn’t improve learning. But that’s what the Indian government is spending their money on. So if I want to help the Indian government on education, I want to test those different things about how the Indian government could improve their education, and then help them reform the education system. And what this set of RCTs has suggested is, not just that it’s about the pedagogy, but it’s specifically about the problem that the Indian curriculum is up here, and most kids are here. If you look at the data, just descriptive data, again, the power of descriptive data… within an average Indian classroom in 9th grade, none of the kids are even close to the 9th grade curriculum. They’re testing at somewhere between 2nd grade and 6th grade. No wonder they’re not learning very much, ’cause the teacher, the only thing that a teacher has to do by law in India is complete the curriculum, even if the kids have no idea what they’re talking about. So yes, you have RCTs testing very specific interventions; all of the ones that worked were ones that got the teaching down from the 9th grade curriculum to a level that the kids could actually understand. Now the lesson from that, the big lesson for the Indian government if they were ever to agree to this, is change your curriculum. That’s the biggest thing that you could do. Reform the curriculum and make it more appropriate to what children are doing. 

This is a staggering level of naivety about how development happens in the real world. I wish I could write in greater detail.

This assessment of China's economic policies has to take the cake,

Well, if there are cases of countries that are as screwed up as China, helping them move to a more effective economic management, that’s gotta be the most effective thing that we could for poverty. You can’t do that as an outside donor, unless someone’s willing to do it.

Yes, this is about a country whose policies over the past three decades have realised the most spectacular economic and social development transformation in all of human history.