The probation problem in India's public sector

Public recruitments come with a one- or (mostly) two-year probation period. The idea is to correct a Type I error (a false positive recruit) in the recruitment process. 

Underlining its importance, the DoPT Master Circular on Probation/Confirmation of 2019 has this to say about probation:

‘Probation should not be treated as a mere formality. The existing powers to discharge probationers should be systematically and vigorously used so that the necessity of dispensing with the services of employees at later stages may arise only rarely.’ 

Unfortunately, while it is part of all recruitment rules, it is widely believed that there are few instances of discharge during probation in any local, state, or central government recruitments.

Before we analyse this, it must be clearly stated that data on this is very patchy in the absence of disclosures by either the DoPT, UPSC, state PSCs, or the central or state government cadre controlling agencies. Empirical evidence, or whatever can be gathered, shows that across every service - All-India Services, central Group A civil, central Group A technical, state civil services, state police officers, state medical services, the great mass of teachers, constables and healthcare workers - the discharge rate on performance grounds is effectively zero. 

This is what Claude gathered about central government cadres.

And this about state government cadres.

It must be disclosed that both the data are not validated, but conforms to the widely known anecdotal knowledge. 

The rare discharges that do occur are almost invariably for fraud (fake caste, disability, TET or degree certificates) discovered post-selection, or for medical/physical failure during pre-probation training. Fraud detection and physical fitness testing are legitimate filters, but they are not the probation function. This has been confirmed by every major civil-service review (Hota 2004, Yugandhar 2003, Second ARC, Baswan 2016). 

There are three examples of probation discharges outside of fraud and medical or physical ineligibility. An estimated 2-5% of the scientists and engineers recruited by the Department of Atomic Energy are discharged for non-completion of the academic curriculum. An estimated 3-15% of the police recruits by state and central government paramilitary forces are discharged for physical and disciplinary reasons (not deficiencies in the acquisition of policing capabilities). Thanks to supervision by the High Courts, an estimated 0.5-1% of every batch of state civil judges and munsiffs are discharged for failure to meet defined output and quality metrics during probation.

The absence of any discharge deterrent increases the stakes associated with the recruitment process itself. This most likely contributes to the fraudulent practices that are pervasive across recruitments at all levels. 

This is a comparative assessment of probation discharges globally and from India’s own private sector. 

The deterrent effect of even a few discharges can be significant. 

So what can be done about this?

The probation instrument has not served its purpose because the framework it operates within provides neither the assessment infrastructure (objective performance criteria linked to a role profile), disclosure requirement (not even DoPT publishes data on probations), nor the political-economy incentive (senior officers who discharge a subordinate invite litigation and administrative-tribunal action). In the circumstances, reforming the probation clause without reforming both the criteria and the incentive structure will change nothing. 

A low hanging fruit is to shine light on the problem and make it mandatory for all departments to disclose the status of probation confirmation, extensions of probation, and affirm that all the probationers met the requisite benchmarks for the same. 

The first step in any systematic effort would be to define a few objective and easily captured metrics of probation performance that are proximate to their roles. This would usher in transparency and shape expectations among probationers about their roles. 

Second, the probation performance evaluation should be made a mandatory exercise, by a committee consisting of the Departmental head (or representative), officer responsible for training within the Department, and the Director of the Training Academy. This would mitigate the political economy deterrent to discharges. 

Third, there could be a mid-way evaluation of the probationers by the same committee, which discusses any laggards and inform them about where they are falling behind. The same should be documented. This could shape expectations and ensure that the probationers are forewarned before any discharge. 

Fourth, the evaluation reports should be submitted every year by the Departments to DoPT (and its state government equivalents) and the UPSC/SPSC, failing which no recruitments by the Department should be allowed. This would bring departmental accountability to the process of probation confirmation. 

The aforesaid measures would constitute a simple and realistic start to addressing one of the most farcical features of the recruitment process. 

AI for organisational and bureaucratic reforms

The debate rages about the extent of AI’s likely impact on the economy and human lives. So far, there has been an apparent lack of commercial value creation to justify the gigantic and exponentially increasing volume of AI investments. 

I blogged here on the distinction between horizontal and vertical use cases of AI, with success on the latter being limited, here cautioning about the likely impact of AI on development and in developing countries, and here on some possible high-impact use cases for AI in lower-income countries. 

The most common horizontal use of AI is in personal productivity improvements. Claude, ChatGPT, etc., are already having large effects on personal productivity. But its translation into vertical use products is muted. 

On this, John Burn-Murdoch points to the work of Mert Demirer, Leon Musolff and Liyuan Yang to make two important points. The first point is the “disconnect between reported increases in coders’ output and the apparent lack of a corresponding boom in product or value creation,” which creates a very steep funnel between inputs and outputs. 

The study by MIT’s Mert Demirer and co-authors tracked software developers’ work before and after they adopted AI tools. Importantly, they measured this at several different levels, from the amount of code written, to the number of discrete files edited, to the number of projects or features worked on, to actual releases of new software. They found an explosive impact at the top of this funnel — coders created or edited almost 300 per cent more files — but that boost was halved to 150 per cent by the time they got to the number of discrete pieces of work submitted for review, and that in turn shrunk fivefold to a roughly 30 per cent uplift in the number of full software releases.

The authors also found little evidence of AI-assisted increases in software development, leading to increased consumption of Apps. 

This brings us to the second point made by the authors about AI’s impact - it is likely to be fully realised only when new organisational structures, markets, and business models emerge.

But Demirer and his co-authors feel the more likely explanation is that current organisational structures and marketplaces are not set up to take advantage of real underlying gains. That view is supported by the evidence from past technological revolutions, where the real jumps in productivity and job displacement came from new companies and processes rather than incumbents grafting new technology on to existing workflows. In the case of electricity in the late 19th and early 20th century, productivity gains were modest where factories simply replaced giant steam engines with giant electric motors but left the rest of the machinery and layout unchanged. The boom arrived decades later when engineers fitted individual workstations with their own small motors.

In this backdrop, Jack Dorsey and Roelof Botha have an insightful article on organisational impact. Specifically, they claim that AI’s productivity-enhancing value can address the fundamental coordination problem in large organisations that manifests in the form of a trade-off between span-of-control limitations (which add organisational layers) and speed of information flows. They argue that AI sharply increase people’s span of control, thereby reducing organisational layers and hastening decision-making. 

The first organisational models emerged in the military to organise large numbers of soldiers into a coherent and effective fighting unit. It involves a hierarchical chain of command that allows for a span of control and a seamless flow of information and instructions. The model then entered the corporate world through the US railroads in the 1840s and 1850s, which borrowed West Point-trained engineers from the US Army. They trace the evolution of the modern organisational form,

In the mid-1850s, Daniel McCallum of the New York and Erie Railroad created the world’s first organizational chart to manage a system stretching over 500 miles with thousands of workers… McCallum’s chart formalized the same hierarchical logic the Romans had used: layers of authority, defined reporting lines, structured information flow. It became the blueprint for the modern corporation… Frederick Taylor (1856-1915), often called the “Father of Scientific Management,” optimized what happened within that hierarchy. Taylor broke work into specialized tasks, assigned them to trained experts, and managed through measurement rather than intuition. This produced the functional pyramid organization - a structure optimized for efficiency within the information routing system that the military had pioneered and the railroads had commercialized… 

In 1959, McKinsey’s Gilbert Clee and Alfred di Scipio published “Creating a World Enterprise” in the Harvard Business Review, providing an intellectual framework for a matrix organization that combined functional specialties with divisional units. Under the leadership of Marvin Bower, McKinsey helped companies like Shell and GE implement these principles, balancing central standards with local agility. This became the “professional” or “modern” corporation that propelled the postwar global economy… The McKinsey 7-S framework, developed in the late 1970s by Tom Peters and Robert Waterman, distinguished the “hard Ss” (Strategy, Structure, Systems) from the “soft Ss” (Shared Values, Skills, Staff, Style). The core idea was that structural elements alone were insufficient. Organizational effectiveness required alignment across cultural traits and the human factors that determine whether a strategy actually succeeds.

They suggest that AI makes it possible to solve the fundamental coordination problem within large organisations that necessitate hierarchical formations. 

For the first time, a system can maintain a continuously updated model of an entire business and use it to coordinate work in ways that previously required humans relaying information through layers of management… In a traditional company, a manager’s job is to know what’s happening across their team and relay that context up and down the chain. In a remote-first company where work is already machine-readable, AI can build and maintain that picture continuously. What’s being built, what’s blocked, where resources are allocated, what’s working and what isn’t. That’s the information the hierarchy used to carry. The company world model carries it instead… 

In a conventional company, the intelligence is spread throughout the people and the hierarchy routes it. In this model, the intelligence lives in the system. The people are on the edge… The edge is where the intelligence makes contact with reality… the edge doesn’t need layers of management to coordinate it. The world model gives every person at the edge the context they need to act without waiting for information to travel up and down a chain of command… Everything else the old hierarchy did, the system coordinates, and everyone is empowered, with a role that’s much closer to the work and the customer.

They identify three roles - Individual contributors (ICs) who are deep specialists and experts who build and operate system capabilities; Directly Responsible Individuals (DRI) who own specific cross-cutting problems or opportunities and customer outcomes; and player-coaches who replace the traditional manager whose primary job was information routing, who do both building and handling people. 

All this makes great sense and points to how corporate organisational models are likely to emerge as the application of AI progresses. There will be frontier firms in a few sectors that will lead the way for others to follow. 

AI applications are a promising opportunity to address inefficiencies and coordination failures in public bureaucracies, too, and improve the quality of public administration. 

For a start, it has the potential to restore internal capabilities, which have eroded steeply. Over the years, thanks to practices like outsourcing all analytical and documentation work to consulting firms and the hiring of individual consultants (most notably now, the system of Young Professionals, YPs, in governments), there has been a complementary erosion of in-house expertise. The capabilities to articulate proposals for internal deliberations and file circulation have atrophied. Given that bureaucracies run on deliberations and files, this trend is an underappreciated aspect of state capability weakness. 

AI provides an opportunity to reverse these trends and develop internal capabilities. The primary reason for the reliance on external expertise is the extent of analytical work and documentation required during the deliberative process (everything from a concept note on the proposal to reports for appraisals, and Cabinet Notes). The bureaucratic leaders who are burdened with a multiplicity of tasks, work under tight timelines, face increased fetters from oversight agencies and courts, and must rely on an increasingly enfeebled internal bureaucracy. In the circumstances, they prefer to outsource the thinking and documentation to outsiders. I have blogged earlier on the perils of this approach. 

AI tools like Claude are excellent at analytical work and the generation of these documents in response to clearly articulated prompts. It becomes a simpler proposition if bureaucrats can quickly and easily obtain a draft concept and supporting documents, and then scrutinise, validate, and refine it before circulation for approval. AI tools can then become a force multiplier for bureaucratic leaders, who are now constrained by their limited bandwidth and acute dependencies. 

This would also empower bureaucratic leaders, or at least some among them, and could enhance the quality of their engagement with the decision-making process. Besides, by minimising the drudgery of the bureaucratic process, it would also allow bureaucratic leaders to apply their minds and exercise judgment more effectively, thereby improving the quality of decisions and policy design and implementation. It would also lower decision-making delays.

It should therefore become a priority of the National Informatics Centre (NIC) (or an AI division within it) to develop or license AI application that is embedded in the e-office software and enables officials to sift through large documents and generate proposals/presentations, circulation notes and reports using prompts. This has transformative potential for productivity improvements, not only stopping the erosion of internal capabilities but also helping rebuild them. 

If this can be done, it opens up opportunities for far-reaching administrative reforms. The current bottom-heavy pyramid can be rationalised to make it fit-for-purpose.

A major inefficiency is the presence of multiple layers within the administrative system. It is a widespread practice across governments to have YPs, and those recruited as data entry operators originate the note file (a task earlier performed by the clerical staff). The note then gets circulated across several layers, often seven or eight till the approver. This can be radically pruned down to no more than three or four, including the approving authority. 

Such de-layering is especially relevant for technical ministries and departments whose activities are more amenable to AI-based support. Such ministries should have a separate administrative staffing plan, one that takes into account the role that AI can play in generating documentation and considerably reducing any drudgery associated with analytical work. 

As a general illustration, there are perhaps three kinds of activities in any department - shared services (HR, procurement, establishment issues, statutory matters, etc.), administration of departmental programs, and analytical and technical work. There are significant low-hanging likely process-efficiency improvements in all three, and substantive value-addition potential in the third activity. 

This would also necessitate a reassessment of public recruitments. The advent of AI applications means that, unlike in earlier times, apart from merely documenting the issues in a note file, the case worker (the ASO or SO) can now be expected to do some analysis and provide comments. This also means that a smaller base can serve the clerical roles (the entire paraphernalia of clerical cadres can be collapsed into just two functional levels - maker and checker), and their educational qualifications and skills must reflect the requirements for the revised scope of work. I’ll blog separately on this. 

The increased use of AI applications to analyse and document, and a compact and delayered deliberative process captured in the file circulation can also increase the quality of collective engagement and ownership of the bureaucracy in decision-making. It lets (and nudges or forces) everyone contribute meaningfully to the process instead of being passive pass-throughs of instructions and note files. It presents the opportunity to shed reliance on outsourced expertise and build back state capabilities.

This is deep work and, even in the best case, is likely to be adopted only by a few units in the first phase. The objective should be to create the conditions that encourage the emergence of these lighthouses and channel them to diffuse change more widely.

Individuals matter, and more so in public bureaucracies

I blogged here about the importance of strong public oversight and in-house expertise for the successful execution and management of infrastructure projects. 

Mainstream development discourse focuses disproportionately on institutional and systemic challenges, and overlooks the important role played by individual officials in effective public services delivery and the realisation of policy outcomes. 

Specifically, I am referring to the commitment and expertise of individual public servants in important positions at all levels of the government. By important, I’m not confining to leadership positions, but any position where they can make significant contributions to influence the agenda. 

The importance of individual officials in the success of development interventions is generally overlooked amidst systemic and institutional factors. This also comes from the belief that the success of development interventions is determined by good ideas, comprehensive planning, and rigorous monitoring. This theory of change overlooks the reality that successful development interventions tend to emerge iteratively over the course of their implementation. Such iteration, in turn, requires capable and committed leadership, especially important given weak state capabilities. 

In fact, it is no exaggeration to argue that capable, proficient, and committed officials are perhaps the most important ingredient of state capability. 

The importance of capable individual officials is also borne out by academic research. I blogged here, drawing attention to the work of Philipp Barteska and Jay Euijung Lee, who examined the impact of the bureaucratic capabilities (of export promotion officers) on the effectiveness of industrial policy in terms of export performance in South Korea. They found the following:

We exploit the three-yearly rotation of managers of South Korea’s export promotion offices in 87 countries between 1965 and 2000 to show that a one standard deviation increase in bureaucrat ability boosts exports by 37%. Under higher-ability bureaucrats, South Korean exports respond more strongly to a country’s import demand, suggesting a more effective transmission of market information.

An increase in exports by nearly two-fifths with just one standard deviation increase in bureaucratic capability tells us that the quality of officials might matter more than (or at least as much as) fiscal incentives and regulatory changes in trade promotion efforts. 

Alessandra Fenizia studied the impacts of managers in the public sector in Italy using a dataset containing an output-based measure of productivity. 

Exploiting the rotation of managers across sites, I find that a one standard deviation increase in managerial talent raises office productivity by 10%. These gains are driven primarily by the exit of older workers who retire when more productive managers take over. I use these estimates to evaluate the optimal allocation of managers to offices. I find that assigning better managers to the largest and most productive offices would increase output by at least 6.9%.

Cristobal Otero and Pablo Munoz study government managers in public health provision in Chile. 

Using novel data from public hospitals in Chile, we document that top managers (CEOs) account for a significant amount of variation in hospital mortality. We then use a staggered difference-in-differences design, and show that a reform which introduced a competitive selection system for recruiting CEOs in public hospitals reduced hospital mortality by approximately 8%. The effect is not explained by a change in patient composition and is robust to several alternative explanations. The financial incentives included in the reform—performance pay and higher wages—do not explain our findings. Instead, we show that the policy changed the pool of CEOs by displacing older doctors with no management training in favor of younger CEOs who had studied management. The mortality effects were driven by hospitals in which the new CEOs had managerial qualifications. These CEOs improved operating room efficiency and reduced staff turnover.

Michael Carlos Best, Jonas Hjort, and David Szakonyi analyse data on public procurements in Russia and find the value of bureaucratic effectiveness. 

Using data on 16 million public purchases in Russia, we show that 39 percent of the variation in prices paid for narrowly defined items is due to the individual bureaucrats and organizations who manage procurement. Low-price buyers also display higher spending quality. Theory suggests that such differences in effectiveness can be pivotal for policy design.

R D Metcalfe, A B Sollaci, and C Syverson

In this setting, managers move between stores but management practices are set by firm policy and largely fixed, allowing us to hone in on managers’ personal roles in determining store performance. We find: (i) managers affect and explain a large share of the variance of store-level productivity; (ii) negative assortative matching between managers and stores, which may reflect both firms’ decisions and a selection-driven bias that we characterize and argue might apply in other settings using movers designs; (iii) managers who move do so on average from less productive to more productive stores; (iv) female managers are less likely to move stores than male managers; (v) manager quality is generally hard to explain with the observables in our data, but is correlated with the ratio of full-time to part-time workers; (vi) managers who obtain high labor productivity also tend to obtain high energy productivity, revealing some breadth in managers’ skills applicability; (vii) high-performing managers in stable growth times are also high-performing during turbulent times; and (viii) exogenous productivity shocks improve the quality of initially low quality managers, suggesting managers can learn. We explain implications of these findings for productivity research.

Ricardo Dahis, Laura Schiavon, and Thiago Scot investigated the performance of state judges in Brazil. 

We investigate this question focusing on state judges in Brazil. Exploring monthly data on judicial output and cross-court movement, we estimate that judges account for at least 23% of the observed variation in number of cases disposed. With novel data on admission examinations, we show that judges with higher grades perform better than lower-ranked peers. Our results suggest competitive examinations can be an effective way to screen candidates.

On a slightly different note, Kevin Hawickhorst shows how technical expertise built up within public systems in the US allowed for the nurturing and flourishing of capable officials, created public confidence, and thereby enhanced the credibility of governments. 

However, over time, expertise has come to be crowded out due to the conscious shift in the way government bureaucracies came to be organised. 

At the turn of the twentieth century, agencies followed a distinct blueprint: they were organized by subject matter, not by abstract function. Each bureau focused on a single domain—such as soils, mines, or forests—and combined research, regulation, and grants under one roof. In the U.S. Department of Agriculture (USDA), the Bureau of Entomology, for example, studied insect-borne diseases, issued rules to contain them, and funded farmers to protect their crops, all as part of a single mission. This structure helped agencies recruit experts by offering broader, more meaningful work than corporations could, and it built a shared sense of mission rooted in a vocational community.

Today’s agencies look very different. After World War II, reformers dismantled the integrated subject matter bureaus and reorganized government along what they called “functional” lines. In this system, regulation is one bureau, research another, and grant administration still another; each bureau covers a wide range of subjects and is defined by its activity rather than its mission. It is the model we now take for granted. The Bureau of Entomology is gone, and USDA now houses all agricultural research in a single unit. New agencies were built this way from the outset: the Department of Housing and Urban Development, created in 1965, was designed as a grantmaking machine, never a vocational community.

The shift was a well-intentioned one and backed by a wide coalition of reformers, businessmen, and interest groups. Functional departments looked modern, rational, and efficient: they simplified charts, tightened chains of command, and promised to reduce duplication. But what seemed like sensible reform gradually hollowed out the structures that had made expertise durable. Once government agencies lost their vocational missions, they stopped drawing on networks of expertise and started looking like paper mills, less able to command political respect, and more vulnerable to capture and drift… logic of the Progressive-era model: that research and administration had to remain intertwined within a unified mission if expertise was to thrive.

However, this wealth of internal technical expertise has, over time, given way to the tribe of generalist managers. 

The core mistake was a shift in what we thought expertise was. The Progressive reformers built vocations that were tied to missions, visible to the public, and legible to politicians. Their successors redefined expertise as a credential: the knowledge of process rather than mastery of a craft. To businessmen and academic reformers alike, competence meant general managerial skill, not professional vocation. As this view took hold within the bureaucracy, “expertise” came to mean knowing the procedures rather than knowing the work. We have traded the civil engineer and the entomologist for the program analyst, the management consultant, and the diversity officer—experts who know how to manage the process but not how to do the work.

This redefinition of expertise hollowed out our idea of representation. We now equate representation with participation and diversity, as if the state were legitimate only when citizens can see themselves in its officials. The Progressives, by contrast, recruited from the country’s varied vocations and made that work visible to the nation. Expertise was representative not because it resembled the public but because it served the public, visibly and competently… Their institutions were built to make expertise endure, by recruiting promising candidates from vocational schools and professional societies, dressing them in uniform, and sending them to work alongside state engineers, agricultural agents, and university researchers…

The Navy cannot build ships. In 1940, faced with the same problem, Congress did the obvious: it created a Bureau of Ships, put engineers in charge, and got ships built. That bureau is gone, and we treat its return as unthinkable. Yet the remedy remains the same. If we want ships, we should once again have a Bureau of Ships to build them… Repairing our institutions will ultimately require returning to the vocational conception of expertise… We have built institutions that valorize process in place of vocation, producing a bureaucracy that neither embodies skill nor commands respect. What matters now is not saving “expertise” in the abstract but rebuilding the institutions where it can serve visibly and credibly.

Hawickhorst’s essay points to several individual public leaders in the US who built institutions and brought credibility and confidence in public agencies through their careers - George Uhler (headed Steamboat Inspection Service for 20 years from 1903), Logan Page (Office of Public Roads, founded in 1905), Joseph Kinyoun (headed the Hygienic Laboratory, a precursor to the National Institutes of Health), and Gifford Pinchot (founded the Forest Service). Every country has such leaders across levels. 

In their search for better outcomes in public policy, governments tend to expend effort and resources on interventions involving financial support, regulatory enablers, and technology adoption, while overlooking personnel choices. This bias is also reflected in public commentary and academic research that shapes public narratives. 

However, as the case of the South Korean export promotion officers starkly demonstrates, for governments intent on reform and impact, personnel choice decisions may be the lowest-hanging fruit. In most policy areas, the range between the opportunity cost of a bad personnel choice and that of a capable personnel choice may be much greater than that for any other policy intervention.

The private sector addresses the issue of the importance of capable individuals by incentivising them with extrinsic material motivations like financial rewards and fast-tracked promotions. While neither of these instruments is available to governments, it can appeal to the intrinsic motivation of public-spirited officials. 

This would require acknowledging capabilities and merit (as borne out strictly by performance track record, not merely in some narrow quantitative sense). This requires differentiating capable bureaucratic leaders from their larger peer group by entrusting them with higher responsibilities, appointing them to identified important positions, drawing on their expertise in various forms, recognising their work through different non-financial channels, and generally signalling their differentiation. 

An explicitly professed intent, let alone a rigorously implemented process, that seeks to differentiate among officials at all levels, can be a powerful force to shape expectations and align incentives within public systems.

Deregulation is rarely a stroke-of-pen reform

There’s a widespread belief that deregulation, as the name appears to suggest, is about the elimination of certain regulations. Eliminate those restrictive provisions with the stroke of a legislative order or an executive decree, and you are all set in the new deregulated world. Unfortunately, while there are some strokes-of-pen deregulations, the vast majority are far from that easy and require sustained engagement. 

Urban planning is a fertile ground for stillborn deregulation. The three commonly discussed planning variables are FAR, height restrictions, and land-use restrictions. Deregulation, as is perceived by commentators, would involve raising FAR and height limits, and promoting mixed-use construction, coupled with measures to ease the process of getting the requisite permissions. But this overlooks several layers of small detail that have the potential to derail any deregulation. 

For illustration, this is the common building rules of a state government. Even without the Annexures, the Government Order itself runs into 26 pages with several details on setbacks, minimum road width, minimum plot size, parking provisions, open spaces, amenities, fire safety and other compliances. This is a consolidation of all the relevant documents and is more than 370 pages long. As can be imagined, the devil is in the details.

It’s therefore not surprising that some Indian cities that claim to have implemented urban planning reforms, including higher FAR and Transit Oriented Development (TOD), have achieved little in substance. One study of a metropolitan city found that onerous details (in terms of minimum plot size and road width requirements) meant that very few sites were able to utilise the liberalised norms on FAR and height. As aforementioned, given the highly detail-oriented context of the reform, notwithstanding its high-minded objectives, it was dead on arrival.

Another example is the Ease of Doing Business (EoDB) rankings. Its biggest failure was its excessive focus on stroke-of-pen changes to laws/rules. The mere enactment of a legislation or issuance of an executive order to change a process was enough to improve rankings, often significantly. The net result was that EoDB resulted in a lot of performative enactments and decrees, with far less substantive improvements in the actual ease of doing business. 

Take the example of the Insolvency and Bankruptcy Code (IBC), hailed as ushering in dramatic improvements in the insolvency restructuring process and contributing to a step change in India’s EoDB ranking. But as the recent Supreme Court judgment on the takeover of Bhushan Steel by JSW shows, effective implementation of the IBC requires addressing the serious deficiencies at the levels of Resolution Professionals (RPs), Committee of Creditors (CoC), NCLT, NCLAT, and the Supreme Court itself. 

The form of an IBC does not automatically translate to the substance of an effective and expeditious bankruptcy resolution. It requires painstaking, long-drawn engagement that complements the iteration and refinement of the law itself with the building of capabilities and ecosystem to ensure effective implementation. 

In general, while there are some such stroke-of-pen reforms, for most changes, the statutory order is often only the first step in a long journey. 

This is a global problem. 

Consider two examples from the UK of the challenges with the effective implementation of deregulation. The Labour government in the UK came to power promising to build aggressively and expand the affordable housing supply. One area of focus is the redevelopment of blighted sites. 

Britain’s cities contain large tracts of brownfield (ie, underused, previously developed) land, thanks to rapid deindustrialisation at the end of the last century. London alone has some 3,500 hectares (8,650 acres). That is around 25 times the size of Hyde Park, and enough space for more than 400,000 homes (London has a target of around 80,000 new homes a year). Clustered by the canals and rivers that were once industrial arteries, the sites are pretty much the only available land in the city. And yet few are being taken on by developers. Building work for just 1,200 new private housing units started in London in the first quarter of 2025, the lowest since 2009 and just 5.5% of the city’s quarterly target, according to Molior, a consultancy. 

But the challenges of building in these sites are immense.

Many borough councils, which largely wield permit power, insist that as many as half of homes in a given development are “affordable”, which immediately rules out smaller sites. At the same time developers are hemmed in by height restrictions and minimum room and unit sizes. From 2026, any building over seven storeys will have to have a second staircase… Some sites, like the former gasworks, require extensive remediation… Ironically, a big problem with ex-industrial plots is biodiversity… Developers must prove that existing biodiversity levels will be increased by 10%, and maintained for 30 years… Such rules illustrate how incentives are skewed. Brownfield developers must go to great lengths to raise the ecological value of derelict, inaccessible sites, often by offsetting. Meanwhile, less environmentally friendly greenfield developments in the suburbs face far lower hurdles.

Another area of focus has been to speed up planning decisions and build on the green belts. But tens of thousands of houses are “stuck in a pipeline because the new Building Safety Regulator is imposing complex design requirements and delaying construction by as much as 18 months.” Then there are mandates on solar panels on all new homes in the spirit of “everything bagel liberalism”. 

Even well-intentioned reforms get caught in the regulatory quagmire that ends up stifling or even killing them. In their book Abundance, Ezra Klein and Derek Thompson write,

“In California broadly, and San Francisco specifically, dozens of pro-housing bills have not led to the construction of more homes, in part because those bills are layered with additional requirements and standards that builders must meet in order to take advantage of the newly streamlined processes. For developers we spoke to, the added costs of compliance weren’t worth it, so the legislation hadn’t led them to build any new homes at all, much less build them faster. The breakneck deployment of wind and solar infrastructure and battery manufacturing has been slowed by outdated permitting and procurement rules that split the Democratic coalition.”

If deregulation is (mostly) not about high-level legislative or regulatory enactments, not one-off enactments, and involves detailed executive orders and painstaking iteration, it’s important that the spirit of deregulation must be imbibed by officials. 

Governments make laws/rules to govern certain activities that must be regulated in the public interest. In terms of the nature of activities being regulated, regulations broadly cover the issue of statutory certificates, payments and benefits (household cash transfers to industrial policy incentives), municipal and utility services (property tax assessment to electricity connections), licenses and permissions (driving licenses to running a school or hospital to consent for establishment of an industry), procurement processes (eligibility requirements to contract enforcement), and generally compliance with existing laws and regulations (Labour Codes to Companies Act). 

These laws/rules have two broad parts: technical guidance and implementation safeguards. The former can consist of a standard (on, say, a technical aspect like safety or efficacy), identification, an eligibility qualification (technical and/or financial) to perform the activity, a legal requirement, or a combination of some or all of these. The latter consists of provisions to prevent abuse of the implementation of the technical guidance (multiplicity of validations). It also includes compliance reporting. While not alone in culpability, many hassles and accessibility problems arise from the latter (implementation safeguard), which applies to the implementation of the enactment. 

I blogged earlier here on many of these issues in brief. 

Every day, government agencies are issuing orders and notifications across central, state, and local governments. Some norms and principles must restrain this process. All such new orders must be examined with respect to these norms and principles. I’ll present a few below whose spirit must be individually and collectively imbibed within the bureaucracy and polity:

1. The first requirement for any new regulation or condition should be a clear and simple articulation of its objective, identification of the stakeholders impacted, and the manner they will be impacted (in terms of their compliance and reporting). It’s not uncommon to find extra layers of regulation creeping in due to a lack of focus on the objective or trying to cover multiple unrelated objectives. . 

2. The second requirement is prudence on the extent of regulation required, which involves a trade-off between objective and practical considerations. 

Consider a product or a technology or a process in the private sector. Their regulatory validation is contingent on meeting some threshold for success. Any increase in the threshold would entail significant incremental costs. The cost-benefit assessment deems this threshold acceptable. This also assumes a certain acceptable likelihood of failure, false negative or false positive. 

However, in public policy, government agencies often tend to frame guidelines to eliminate any abuse. This leads to tight gatekeeping and access requirements that invariably end up detracting from the objectives. It manifests in the form of enhanced eligibility requirements, additional documentation and certifications, physical verifications, etc. To prevent the likelihood of abuse by 1%, the remaining 99% are penalised with the additional implementation safeguards. 

One way to address this problem would be to have a mechanism that requires officials formulating the safeguard to necessarily examine and trade-off between the elimination of abuse and harassment of the stakeholders in an explicit manner, and then make a choice. 

3. A third requirement should be that the compliance criteria should be defined with clarity, without leaving it open to interpretation. The flexibility to exercise discretion in the interpretation of a regulation, especially in high-stakes issues, is a recipe for harassment and corruption. 

4. A fourth requirement is that the regulation must be formulated with the least burdensome and lowest cost path to achieve the objective. So if there’s an alternative formulation that meets the objective and is less burdensome (or invasive), the same must be preferred. 

5. If a regulation/condition is difficult to define and/or monitor and/or enforce, it’s better to eliminate it (if existing) or not enact it at all (if newly proposed). For example, the assessment of the income of a household to issue an income certificate is fraught with problems. Similarly, the requirement of setbacks on small plots (say, less than 200 sq yards) is most often violated and engenders perverse incentives.

6. If a criterion or compliance requirement is so onerous as to be impossible for compliance by all but a few, it’s best avoided. It should be replaced with a second-best compliance requirement. 

So, for example, if testing facilities are too few, it’s impractical to mandate the criterion/standard. Or, where compliance reporting burden/cost is prohibitive in terms of transaction costs and can be monitored with reasonable certitude through governance interventions like random sample audits, they should be preferred. Another option is to accept self-certifications and supplement them with random sample audits to ensure deterrence, depending on the stakes involved. 

7. The uniform application of a regulation that’s primarily intended for a subgroup must be avoided. For example, if one subgroup poses a risk, it’s best to confine regulation to that group rather than have it applied to everyone. It’s best to have targeted regulations, or have differentiated regulations appropriate for each subgroup, or use governance mechanisms to regulate the subgroup. 

8. Governments tend to respond to emerging reports of abuse of the provisions of a law by incorporating additional safeguards that act as a new layer of regulation. This should be done with caution, since while the new safeguard will likely limit the abuse by those few, it will also increase compliance burdens for everyone. 

Therefore, as a default, the abuse of a system should be addressed through better governance instead of regulation. Such governance would involve more rigorous monitoring, use of data analytics, digital workflows etc., without adding a new regulatory/compliance layer. 

9. On a related note, in general, a very high standard of scrutiny must be applied for any proposal to add to or tighten an existing condition/regulation. They should have a compelling justification that’s recorded by the competent authority. 

10. If there are significant and quantifiable costs associated with the regulation, it’s useful to quantify and undertake a cost-benefit assessment. If the stakeholders must bear these costs, it’s useful to also examine how it would impact them (for example, in the case of a business, its business model). 

11. Finally, as a principle, the incorporation of any new regulatory/compliance requirement should be accompanied by the easing out of two old requirements.

All of the above can be consolidated into a checklist that can be applied to screen any new regulation/condition that imposes a compliance on an individual or company. Foremost, can the objective be achieved by some other mechanism, which is less invasive or burdensome? Can compliance be monitored and enforced? Is the process for compliance easy and simple? Is the reporting of compliance easy and simple? Are the abuse safeguards onerous? And so on. 

On the same lines as for new compliances, any reform involving deregulation should be subject to a similar test on implementation. Does the deregulation achieve its objective in practice? Are there implementation details that are likely to derail its applicability? And so on. 

It may be useful for governments to consolidate these principles and issue them in the form of executive directives to guide the formulation and implementation of regulations. 

What constitutes development administration?

This post is meant to be a summary of several posts on development. It seeks to illustrate the fundamental issues of development by examining the activities and tasks of four departments. 

Consider the fundamental question facing government officials in a few departments.

What’s required to achieve student learning outcomes and create a skilled and employable workforce? What’s required to deliver good quality public health and sanitation to prevent epidemics, maternal and child health, and diagnostic-cum-treatment services? What’s required to improve crop productivity and raise farmer incomes? What’s required to ensure hassle-free access to good quality municipal facilities and services?

Education, Health, Agriculture, and Municipal Administration officials must figure out the answers to their respective fundamental challenges and then execute them.

Let’s use a framework to analyse each sector. Government departments apply inputs, including those from programs and schemes, and undertake a set of tasks. The department’s theory of change is that their effective combination would lead to its desired outcomes.

Accordingly, Figure 1 lists the school education department's inputs, programs and schemes, and tasks.

Figure 1: School Education Department

Figures 2-4 do the same for Healthcare, Agriculture, and Municipal services.

Figure 2: Primary and Secondary Health Department

Figure 3: Agriculture Department

Figure 4: Municipal Administration Department

While these are just four samples, they form a major part of the development landscape. Their features broadly represent development in general, across developed and developing countries.

An examination of the lists in each case reveals some important insights. One, despite all the changes in the world around us, the lists of inputs, programs, and tasks in each sector are both universal and have remained the same over a long time. Two, any program and scheme generally contribute some of the inputs to the departmental tasks. Third, a set of activities/tasks combines all the inputs and programs in some manner as specified (based on the theory of change) in their implementation guidelines. Fourth, the effectiveness of implementation is critically dependent on how these activities/tasks are executed. Finally, it’s assumed that all these activities/tasks can be executed effectively through the standard bureaucratic administration, with its associated monitoring and supervision.

Given their centrality to effective public service delivery, it's useful to examine closely the nature of these activities/tasks performed by departments. Four points come to mind.

For a start, close examination would reveal that most of these activities/tasks are basic enough, but complicated by their interaction with the context and the dynamics that emerge from that interaction. Two, the nature of these activities/tasks is such that they generally require high engagement by government officials. In other words, the quality of the performance of these activities/tasks depends on the quality of engagement by the relevant officials.

Three, related to the previous point, there’s only so much that improvisation and technology can do to commoditise or simplify the activities/tasks such that they can be delivered without compromising quality. Finally, notwithstanding this limitation, the effectiveness of implementation can be enhanced, mostly only at the margins, with improvisation and innovation like process reforms and the use of digital and other technologies.

All these have some important implications for development thinking.

One, the scope or possibility for new programs or improvements to the design of existing programs (or generally new ideas) that can significantly improve outcomes is limited. Two, instead, the primary role of innovations and new ideas would be to improve the fidelity of implementation. But there are clear limits to how much they can contribute. They cannot cover up for deficiencies in governance and state capabilities. Three, the primary focus for the administration of these departments should be on the effective execution of its activities/tasks and its programs and schemes. This is all about implementation by government officials through public institutions.

Fourth, in the circumstances, the primary role of evidence is to enhance the effectiveness of implementation. This means administrative data, surveys, and qualitative feedback are important instruments. Finally, given the nature of these activities/tasks, the effective implementation of many of them involves problem-solving, iteration and adaptation, management of people and activities, and exercise of good judgment. All these are activities/tasks that require high individual and institutional capability levels. 

Management theories and public systems administration

In a recent speech, N R Narayanamurthy, the co-founder of Infosys, suggested that recruitment to the Indian Administrative Service and Indian Police Service should be done from Business Schools rather than through UPSC examinations. 

“It is time for India to move from an administrative mindset to a management mindset. The administration is all about the status quo. On the other hand, management is all about vision and high aspiration. It’s about achieving the plausible impossible,” he said. According to the Infosys co-founder, the current system of competitive UPSC examinations can only produce civil servants trained in general administration. He recommended a management-based approach that focuses on vision, cost control, innovation, and rapid execution to cater to the changing demands of governance.

This comment comes even as Donald Trump has enlisted Elon Musk to lead the new Department of Government Efficiency in his administration to cut inefficiencies in government (substitute for removing extra staff and deregulating processes).

The underlying premise is that public bureaucracies are inefficient in terms of bloated staff and being mired in red tape. They could be improved by importing management practices taught at business schools and widely applied in the private sector. This is the latest reprise of a well-versed cliche with a long history. 

Narayanamurthy and Elon Musk stand on the same side in their deep ignorance of the nature of the activities of governments. Let me try to explain. 

Google AI search informs that the core principles of modern management consist of division of work, authority and responsibility, discipline, unity of command, unity of direction, subordination of individual interest, and remuneration. Management 101, as applied in the private sector, essentially consists of enhancing efficiencies and hastening decision-making through process re-engineering and system transformations, minimising costs primarily by shrinking staff, and improving execution by selecting the right people and aligning incentives among them. 

I can think of several insurmountable areas of divergence between these management theories and the challenges of actual policy implementation.

1. The core activity of government involves running large and dispersed institutional networks, to deliver statutory (Tahsildar, police, and regulatory offices) and non-statutory (schools, hospitals, anganwadis, municipal, welfare etc.) services, through officials recruited on a lifetime employment basis and who are deeply enmeshed in the local political economy. 

These core activities of governments across sectors have hardly changed over time. Neither have the methods and instruments available to them to implement and administer those activities. The problem has been the state’s failure to ensure the effective implementation of those basic sets of activities. This arises from state capability deficiencies and governance failings. 

There’s a belief, drawn from the private sector, that public sector problems can be addressed through innovations. But as I have blogged here, it’s misleading to assume that we can leapfrog fundamental state capability deficiencies and governance failings and innovate (or digitise or privatise or outsource) the way out, as is the practice in the private sector. Instead, there’s a need for boring and painstaking work of building capabilities and delivering good governance. Management 101 is unlikely to be of much value in this endeavour. 

2. Since governments are trustees of public interest and use public resources, strict procedural safeguards in public-sector decision-making processes are unavoidable. This is especially desirable in weak institutional systems like in India, which are prone to corruption and capture by vested interests. This places inherent limitations on the freedom, flexibility, and speed of decision-making. Besides, decision-making in a political system involves tight coordination between the bureaucratic and political executives. Such coordination happens through institutional processes and rules that further constrain decision-making freedom. Furthermore, important public sector decisions invariably require the mobilisation of electoral support, something which is outside the control of even the political representatives. 

These constraints are inherent to public systems across developed and developing countries and have remained so despite all the social, technological, and other changes over time. The private sector is not constrained by any of these factors. 

3. Management 101 extols the wisdom of allocation of roles and responsibilities and delegation of powers. However, in public systems, such delegation tends to backfire. The underlying premise is that people once appropriately empowered or authorised are both capable and incentivised enough to fulfil their responsibilities, failing which they can be replaced. These assumptions do not hold with public systems. Complicating matters, measurement or attribution is a challenge with their activities, thereby making the enforcement of accountability very difficult. 

Therefore, in public systems where capabilities are weak, most often it’s required to supplement managerial work allocation with direct monitoring of the frontline. At the least on critical tasks, leadership must cut through layers and engage directly with the frontline officials to both limit transmission losses in instructions and ensure reliable feedback. 

Management 101 would argue that such direct engagement will weaken the chain of command and distort incentives. And it does. But without it, it’s most likely that execution in public sector contexts will flounder. 

4. Traditional management theories can break down when faced with the management of government employees. For a start, organisational leaders cannot select their team (or even deputies) and must work with those available (or unavailable) in systems where incentive misalignment has uncontrollable contextual roots. There are deeply constraining limits to disciplining, let alone removing people. Even the standard role-allocation and authorisation-based management strategies are blunt in systems where meaningful performance accountability and its enforcement are, at best, extremely challenging and normally impossible. 

5. Management 101, following the pervasive trend in the private sector and public sector in developed countries, similarly advocates the virtues of outsourcing work to consultants and third-party service providers. In public systems, such parcelling out work is unlikely to be effective for multiple reasons - inherent difficulties in disaggregating core activities, limited or weak supply side for these services, and difficulties in monitoring the quality of service delivery. 

6. There’s very little that management theories can teach us in managing relationships with politicians, media, and civil society groups. The very wide variety in the features of these relationships and their interaction with the social and cultural norms of the contexts makes them ill-suited to templates or cookie-cutter approaches. They require good judgment that’s heavily influenced also by the specific context and circumstances. 

7. Finally, development contexts exhibit a very wide variance across people, tasks, and situations. This also means that management in such contexts will have to go beyond templates and theories and involve the exercise of judgment, one which emerges from experiential learning. In many respects, public sector management is a specialisation in itself.