J-PAL's latest big RCT covers the impact of independent, randomly assigned environmental auditors on polluting firms. The reform,
Before the reforms, firms chose and paid their auditors directly. There was also no mechanism to scrutinize the quality of auditors' reports. As a result, auditors that reported the truth were unlikely to be hired, especially by highly polluting firms that did not wish to be noticed. The experiment increased auditor independence. In a pilot group of nearly 500 plants, the researchers randomly assigned some firms to maintain the status quo audit system and others to come under a new scheme where auditors were randomly assigned to plants, paid from a common pool, and had some of their field work independently double-checked. The double-checking enabled the researchers to compare audit reports to the true underlying pollution levels at industrial plants... Under the new system, auditors were 80 percent less likely to submit a false pollution reading.
The results,
Auditors systematically reported plant emissions just below the government standard, although true emissions were typically higher. For example, for particulate matter—a harmful air pollutant—auditors reported that only 7 percent of industrial plants violated the government standard when in reality 59 percent were emitting more than the standard. The new audit system led auditors to report pollution more truthfully and substantially lowered the number of plants that were falsely reported as compliant with pollution standards. In the restructured auditor market, the accuracy of auditor reports increased significantly: the auditors were 23 percentage points (80 percent) less likely to falsely report a pollution reading as in compliance with the relevant regulatory standard. Additionally, their reported pollution readings were 50–70 percent higher than those of auditors working in the status quo system. Industrial plants reduced pollution in response to more accurate audits. Plants facing the new audit system reduced pollution by 0.21 standard deviations on average.
Three observations
1. I am not sure whether it required a long and expensive RCT to find out that audits done by randomly-assigned, independent (in selection and payment) auditors are far more effective than those by auditors selected and paid by the very firms they are auditing. It is only surprising that such audits were even being allowed to happen. The straw-man (business as usual in the Pollution Control Board, PCB) is therefore a very bad, even disingenuous, baseline. In fact, independent and randomly assigned audits are commonplace in some states for smaller engineering works, like the school buildings under SSA or Panchayat works.
2. It is certain that this strategy would have cropped up in audit process reform discussions within the PCB many times over. The J-PAL study played the role of credibly and objectively illustrating the advantages from the reform, thereby creating the momentum required to overcome the internal resistance. It goes back to the argument that managements most often hire reputed consultants to provide internal credibility to reforms they want to implement. The PCB found the study a convenient handle to push through the reform. But this begs the question as to whether the same could have been done more quickly and cheaply through a deep-dive problem-solving by hiring, say, IIM Ahmedabad.
3. Finally, even assuming one gets the audit implementation design right, there is the question of scale-dynamics. Here we run into questions about effectiveness of supervision and state-capability. This long series of events - communicating the right inspection formats, transparently selecting auditors, randomly-assigning them, ensuring their effective inspections, managing the secondary-checks, consolidating all reports, scrutinizing them, reporting for action on violators, initiating action, and persisting with proceedings till punishment is imposed - are most often beyond the scope of the enfeebled state machinery.
Accordingly, officials turn a blind-eye to even egregious transgressions. In fact, it is not uncommon for such reports to pile up, without anyone even reporting on them, leave aside initiating action. Further, once scaled up, the supply-side constraint too kicks in - only a handful of auditors have the capacity to credibly transact their inspections. Furthermore, it is only a reflection of the weakness of state capability, evident in the quality of monitoring, that such egregious violations were happening. It may be a stretch to expect the same state to now act dutifully on the inspection reports.
No comments:
Post a Comment