The challenges with exercising good judgement

I have blogged earlier about decision-making as essentially an exercise in good judgement.

Such judgements face formidable challenges. For a start, most often agents face incomplete information and have inadequate experience. After all good judgement is about inductive reasoning - draw on data points or experiences and draw generalisable inferences. Second, agents are captives of their own prejudices and preferences, which brings in bias to their judgements. Such biases can offset even experience and information.

Finally, such judgements, even when processed based on comprehensive information and free of biases, are inherently inconsistent, with wide variances, something which Daniel Kahneman et al have described as noise. They write,

Professionals in many organizations are assigned arbitrarily to cases: appraisers in credit-rating agencies, physicians in emergency rooms, underwriters of loans and insurance, and others. Organizations expect consistency from these professionals: Identical cases should be treated similarly, if not identically. The problem is that humans are unreliable decision makers; their judgments are strongly influenced by irrelevant factors, such as their current mood, the time since their last meal, and the weather. We call the chance variability of judgments noise. It is an invisible tax on the bottom line of many companies.

Some jobs are noise-free. Clerks at a bank or a post office perform complex tasks, but they must follow strict rules that limit subjective judgment and guarantee, by design, that identical cases will be treated identically. In contrast, medical professionals, loan officers, project managers, judges, and executives all make judgment calls, which are guided by informal experience and general principles rather than by rigid rules. And if they don’t reach precisely the same answer that every other person in their role would, that’s acceptable; this is what we mean when we say that a decision is “a matter of judgment.” A firm whose employees exercise judgment does not expect decisions to be entirely free of noise. But often noise is far above the level that executives would consider tolerable—and they are completely unaware of it.

Their conclusion,

Where there is judgment, there is noise—and usually more of it than you think.