Policing is hard - a bit of humility would be great!

I really cannot understand the purpose of this oped. Abhijit Banerjee points to the increased use of breathalysers (tools) and introduction of higher fines (laws) as essential in the fight against drunken driving. This, he says, has to be complemented with a strategy of vehicle checks at randomly decided locations using dedicated teams of police drawn from the reserves.  

But there is nothing new about this "strategy" that Banerjee claims to have figured out with an RCT. In fact, it is exactly this "strategy" that police superintendents and commissioners resort to for short periods when something (usually a high-profile accident or a court directive) precipitates greater vigilance on drunken driving. And some of the more enterprising police leaders with interest in traffic issues too adopt randomised checking locations strategy. The problem is that these things cannot go beyond short periods. Coincidentally, they are also exactly the same "strategies" than new entrants and popularity seeking police leaders embrace. 

Take the issue of random locations. While, from the outside, allocating random locations may appear an algorithmic exercise, it is far from that when operationalised in scale. A combination of closing ranks by powerful and entrenched interests and systems with very weak institutional capacity at field (read police station) levels mean that such strategies are easily sabotaged, unless there is a strong enough leader micro-managing the process. As an example, random third party quality checks of engineering works under construction, now passe, has become completely captured in many jurisdictions. As to reserves, their deputation beyond a few days, is by definition, impossible. In a heavily under-staffed police force overburdened with crowd and VIP events management responsibilities (bandobasth), reserves, are that only in name. 

In fact, the optimism with even with the first two may be misplaced. The challenge, as Esther Duflo pointed out in other contexts, is with the plumbing. We need to understand the evidentiary standard for a legal offence of drunken driving. For example, in order to limit discretionary excesses, the law (regulations issued on the central Act in different states) mandate that breathalyser tests have to be carried out in the presence of a police personnel above a certain rank. And over-burdened policy administrations have too few personnel of such rank to spare for any traffic responsibilities, much less for night-time drunken driving patrols. But delegating this responsibility raises its set of problems.

Actually, I can unpack several plumbing layers which can make the challenge even more daunting. But this is sufficient for our purpose. 

The article itself is drawn from this research paper which conducted RCTs covering 162 police stations in Rajasthan covering five management interventions - limitations of arbitrary transfers, rotation of duty assignments and days off, increased community involvement, on-duty training, and "decoy" visits by field officers posing as citizens. It found the first three "which would have reduced middle managers' autonomy, were poorly implemented and ineffective", while the last two had "robust impacts". Based on these findings, the researchers found "very large outcomes" from an intervention that linked "good performance" on sobriety tests without relying on middle managers to  the "promise of a transfer from the reserve barracks to a desirable police station posting".

The paper's headline finding is plain misleading, maybe even dangerously populist,

The experimental results in this paper show that it is possible to affect the behavior of the police in a relatively short period of time, using a simple and affordable set of interventions.

If only it were possible to easily replicate a sanitised pilot! So what do the authors conclude,

It is striking that two of the three most clearly successful interventions had never been advocated by the many police Reform Commissions or in the discussions we had with the police top brass despite the motivation for reform and support of the top administration. In contrast, the interventions that failed to work were all carefully selected by the police leadership, partly based on the recommendations of various Police Reform Commissions, who among them combine a huge amount of experience and expertise. These senior police officers also did not lack human capital, being selected through an extraordinarily competitive set of exams. Yet they had clearly substantially underappreciated the difficulty of implementing these interventions even with their full backing, suggesting that they did not fully realize the nature of the informal authority enjoyed by the station chiefs.

And this,

We see the relatively poor performance of police force interventions as evidence against the view that the managers know how to translate the general principles of management into solutions that are relevant for their organization. The managers in this case were the elite of the Indian police, with decades of experience and who were motivated enough, among other things, to engage with us to launch this reform program. They had a clear and articulated understanding of the principles that motivated the interventions and they were recognized as outstanding leaders. However, none of that could guarantee that they would automatically hone in on all the right interventions. And while they clearly shared the view that incentives, as a general principle, are important, they could not see a way to introduce them within the political and administrative constraints, until we intervened as pseudo-consultants. What do outsiders, with less institutional knowledge and experience, provide in these situations...

Really! This is staggering hubris, even if borne out of ignorance.

Let us examine the six (five plus one) "strategies" that the researchers have explored. "On-duty trainings" are a staple of any administrative system. The problem is just that the trainings do not get translated into meaningful enough learning or internalisation, even when done on the right set of issues and by outsider consultants. "Decoy" visits by field officers is more a figment of juvenile imaginations from stories of kings visiting their kingdom in disguise to obtain citizens' feedback than an institutional solution. In fact, if at all police leaders want better feedback, instead of hiring decoys, they should hire the services of third party agencies (and manage it well) or randomly solicit telephone feedback through call centres from people who have just interfaced with the police system.

Good police superintendents and commissioners (is actually true of managers at any level) do not require fancy "decoys" to quickly figure out more or less what is going wrong and who among their officers can be trusted. The practical difficulty of identifying and managing the activities of decoys is immense. Instead of trying to improve the effectiveness of their institutionalised intelligence wings, sending out "decoys" is exactly what populists and band-aiders would suggest. Finally, the researchers would not even be able to imagine the ways in which such "decoys" can end up creating an even bigger problem for the police force.

Take the case of the last intervention, linking transfers to good performance. Again plumbing is the challenge, which the researchers have unsurprisingly glossed over. 

For a start, drunken driving enforcement would hardly figure among the top-ten priorities of the mainstream police force, leave aside of the reserve force. And their (reservists) main bandobasth activities are not amenable to quantitative assessments of individual policemen. Second, how sustainable is a policy that explicitly seek to reward some policemen with transfers to "desirable" places (for whatever "good performance") and penalise some others (the natural corollary) by drafting them to reserves? 

Third the moment we start linking incentives to quantitative performance indicators in detecting drunken driving, it is only time before we get into targets and slip down an undesirable slope. Four what is the sustainability of an administrative process where there is no involvement of middle-managers? Ultimately managers at some level have to be managing this institutionally. Even assuming that level exists outside the "middle-managers", are we any less likely to have concerns with them? And is it even practical to think about such administrative stuff without the involvement of middle managers? Finally, it is far from true that "evaluation generated evidence and information is not typically available to the police leadership". "Evidence", of a far more sweeping breadth and with more than the requisite credibility and rigour, in large measure, is available, to police leaders who keep their eyes and ears open. No amount of careful evidence generation can get you beyond a few baby steps in your endeavour to effectively manage large systems. 

The authors of the Reform Commission reports, in contrast to academic researchers, are life-long plumbers, who, with varying degrees of success (or failures), have grappled with the plumbing challenges of policing in the real world. They were spot on with the three recommendations, which, however you look at police reforms, cannot but not be a part of any end-stage that Rajasthan Police would hope to achieve with police reforms. In fact, they are essential plumbing necessities in any administrative system, all of which can be very logically established (not being done here because of lack of space). In contrast, the three researchers' solutions, as discussed above, suffer from serious practical deficiencies. The Reform Commissions refrained from such band-aid recommendations because they were responsible and honest (apart from not being ignorant) to not do so.

Instead of suggesting solutions that help improve institutions and systems that are at the cutting-edge of policing activities, the researchers end up recommending piece-meal and unsustainable solutions. The objective, for example, should have been to improve policing outcomes by enhancing accountability of middle managers and enhancing the quality of intelligence from institutional channels, instead of hiring decoys and dispensing with middle managers.

This example is teachable in many respects. Abhijit Banerjee acknowledges the importance of  the so called inputs and regulations - breathalysers and prohibitive enough punishment as critical deterrents. But he also points to the role of complementary factors (strategy) to the success. 

It is in the same vein that we should see other input and regulation prescriptions in cases of education, health care, nutrition, state capacity and so on. Everywhere inputs and regulations are critical requirements to even get to the starting point. While they are necessary, they are not sufficient. Complementary levers of implementation as essential.

I have written about this many times earlier and let me repeat it. Performance incentives that involve large enough financial rewards or transfers are unlikely to work in scale in weak public systems like in India. After all where does it work in even developed systems? For a start, several plumbing challenges, mostly related to weak state capacity and the socio-political environment, increases the likelihood of an even more inefficient general equilibrium. Second, quantifying outcomes in a credible manner is deeply problematic, and collecting and managing it even more so, in most contexts. Third, financial incentives most likely end up becoming entitlements, thereby worsening the problem of already high lower level government salaries. Finally, transfers are among the highest stake administrative actions, and when done in environments where rank ordered "good performance" cannot be credibly and indisputably established is a recipe for controversies.

Let me state this and I would be happy to be refuted by practitioners from the police bureaucracy. There are no innovations, either great ideas or process re-engineering or even management theories, that can dramatically improve policing outcomes in conditions of acute systemic and leadership weaknesses. Wherever police systems work well, it is more likely a combination of functional administrative capacity and good leadership. The intensity of the latter can even temporarily mask deficiencies in the former. It is for this reason that we keep hearing stories of poorly run administrations suddenly becoming efficient with the arrival of a good Police Commissioner.

In economists' speak, the production function for good policing outcomes is largely these two. Innovations most often can work at the margins to improve administrative capacity and free up leadership energies for productive use elsewhere. But in really weak systems, as we have here, leadership is necessary to both generate short-term good outcomes and build long-term institutional capacity.

That this paper got through a peer-review is a reflection of how badly disconnected the real world of experimental economics is despite at least some economists' claims to being superior plumbers. These are thoughtful economists who swear by the general equilibrium. But that, in turn, requires deep plumbing knowledge. Not exactly their forte, despite claims to the contrary.