Context matters, but are the economists listening?

In a famous 2007 paper, Robert Jensen, heralded the case for the use of mobile phones in advancing development outcomes. He used microlevel survey data on the adoption of mobile phones by fishermen in Kerala over 1997-2001 to show that,

the adoption of mobile phones by fishermen and wholesalers was associated with a dramatic reduction in price dispersion, the complete elimination of waste, and near-perfect adherence to the Law of One Price. Both consumer and producer welfare increased.

Never mind association is not causation, mobile phones could bridge all kinds of information asymmetry and so on, the story got constructed. Everyone related played along with the narrative. Since then on countless instances, this paper's findings have been invoked to make the case for use of mobile phones in development interventions, and that too a very narrow category of uses. 

Fast forward 2015. Janaki Srinivasan and Jenna Burrell do ethnographic research that reveals a more nuanced explanation for the outcomes seen,

We argue that in this case, a parsimonious model (that omits details such as the existence of fishers’ cooperatives, the mediation of fish sales by investor-auctioneers, and the structure of investments in the industry) has skewed and narrowed the solution space in particular ways, thereby framing what policy or design implications are likely to be extracted from the study. Our broader concern is to understand the consequences of a single study disproportionately influencing our thinking in a field such as ICTD. An important way in which Jensen’s work narrowed the solution space was through its deployment of broad categories such as “fishermen” or “buyers.” These categories concealed considerable diversity in economic and social circumstances. Over time, as mobile phone ownership has increased, diverse ways of using the mobile phone have emerged within these broad groups. In this way, we critique the reliance on broad categories and on aggregates and averages that generalize the needs, positive uses, and outcomes of the relatively more affluent to the whole category. Such a reliance can obscure the potentially different requirements of lower-income subsets of the larger group. The qualitatively distinct uses of the mobile phone by (for example) small vendors, small-scale fishing units, and workers without boat-owning shares were often related to managing risk and vulnerability rather than optimizing efficiency. These uses would suggest a different class of design interventions entirely, ones that seem (so far) to be little attended to in ICTD-related work...

A second example of disciplinary skew from Jensen’s work is its considerable emphasis on the power of impersonal information relative to the social basis of information circulation in trading relations. We found that while information was by all accounts critically useful to the Kerala fishing industry as a whole, its utility clearly flowed along a baseline of exchange relationships developed among actors in the supply chain. Jensen’s findings about information availability, mobile phones, and market efficiency, while intended as a contribution to debates within economics, are regularly enlisted to justify larger policy or technology design decisions and have given credence to widespread but (we would argue) misguided solutions involving impersonal information provision. For example, numerous SMS-based price information systems and services such as Esoko, Nokia Life, and Reuters Market Light, which are now available in many countries, are built on the assumption that information is actionable in isolation. These systems misunderstand the enduring power dynamics of trade and its basis in person-to-person relationships that we found in north Kerala.

Finally, the depiction of the market in Jensen’s account as seemingly more open and free than our further investigation revealed has provided weight (in the policy context) to notions of pure technology transfer (absent market regulation or other reforms) and the adequacy of private-sector solutions (the latter Jensen calls out specifically in his conclusion as an implication of his study). However, we show that the systems of auctioning and investment that beach markets relied on were brought about by the efforts of both state and nonstate actors. Fishing regulations and rules were also enacted and implemented by the state. Far from complaints about interference from the state, the fishers we met complained only of how the state was not doing enough to police these regulations. On the balance, we found that the fishers’ cooperatives and the auctioneering system are one type of solution that was successful by all accounts in enhancing fishers’ profits. The introduction of mobile phones was another. In this context, both were necessary to yield the observed improvements in welfare. A representation of a market that does not account for these ways in which it has been regulated has political implications. This is of special concern in ICTD where claims about the empowering qualities of information or of ICTs can make underlying and ongoing political struggles invisible. Yet, these struggles shape the market and the power relations within which ICTs can be effectively put to use.

In fact, the failings of the Jensen paper raised above are applicable across sectors where digital technologies are now being proposed as solutions by researchers for a wide variety of development challenges. No matter the context. No matter whether the information is actionable.

The Jensen paper used survey and some other data to make the simplistic causal inference about the efficacy of mobile phones. It overlooked the importance of Kerala's unique context - coastal geography, fishermen's co-operatives, intermediaries, credit relationships etc - which allowed its fishermen to make good use of mobile phones.

This is symptomatic of the largely superficial understanding exhibited by international development  experts, especially economic researchers schooled in rigorous quantitative techniques and with limited exposure to economic history and inter-disciplinary engagement, much less contextual experience.

It is common place for economists to undertake RCTs and data-focused research on a variety of development and social issues without too deep an understanding of the specific context in which those issues are situated. Such research does not sufficiently appreciate the underlying contextual factors that contribute to those issues; the cultural, social, and political consequences of the reform/intervention being proposed; the multiplicity of theories of change, and so on. The failure of economic models to account for real factors is widely discussed - inflation is caused by real factors and not just money supply, investment is motivated by real factors and not just low interest rates, and so on.

Another critique of the Jensen paper is here and here.

This also underlines the limitations of pure quantitative evaluations, including RCTs (in failing to decipher the blackbox), the importance of bringing it down from its current pedestal of being the touchstone for evidence, and focus instead on other more comprehensive and versatile forms of evidence, including rigorously done ethnographies. See Martin Ravallion and Jean Dreze.

Interestingly, the Srinivasan-Burrell paper was published in Information Technologies and International Development, and not any major Economics journals! And I have not come across any discussion that questions the original Jensen paper in light of this paper. A case of lack of intellectual honesty within the profession?