In a new book, Noise: A Flaw in Human Judgment, Daniel Kahneman and co-authors study professional judgments made in hiring, sentencing, insurance underwriting, medical diagnosis and many more fields -- not including cyber and technology risk management – and land in about the same place as Factor Analysis of Information Risk (FAIR™): Unexamined, qualitative “mental models”, as FAIR creator Jack Jones calls them, get us into a lot of trouble.
Learn quantitative analysis of cyber and technology risk – take FAIR Fundamentals Training through the FAIR Institute.
Kahneman won the Nobel Prize in economics for his work explaining the psychology of decision-making. In the new book, “Noise is variability in judgments that should be identical” when made by “interchangeable professionals” – let’s say two cyber risk analysts, one rating a risk as “yellow”, another rating it as “red”.
“In many areas, the level of noise is too high. It is imposing high costs and terrible unfairness.”
What causes noise? Kahneman identifies one situation familiar to many FAIR cyber risk analysts and managers. “Research in managerial decision-making has shown that executives, especially the more senior and experienced ones, resort extensively to something variously called intuition, gut feel or simply judgment…This emotional experience (‘the evidence feels right’) masquerades as rational confidence in the validity of one’s judgement (‘I know, even if I don’t know why’).”
Unfortunately, “judgments of one’s ability to make precise predictions…are notoriously overconfident.”
How to Quiet Noise for Risk Managers and Other Decision-makers
The book recommends that organizations put themselves on a regime of “decision hygiene” by enforcing several techniques that will sound familiar to FAIR practitioners, among them:
- “Think statistically and take the outside view of the case”
To reduce “gut feel” thinking on the case at hand, stay focused on the range of outcomes of previous similar cases.
- “Structure judgments into several independent tasks”
Decomposing risk into factors is the essence of Factor Analysis of Information Risk, of course. Kahneman says this technique combats the psychological mechanism of “excessive coherence” that causes people to downplay information that doesn’t fit a pre-existing or emerging storyline.
- “Favor relative judgments and relative scales.”
FAIR analysis produces results as a range of probable outcomes. That turns out to fit well with decision psychology. “Relative judgments are less noisy than absolute ones, because our ability to categorize objects on a scale is limited while our ability to make pairwise comparisons is much better.”
Kahneman also makes a useful distinction between rules and standards in decision-making that runs parallel to the spirit of FAIR.
A rule might be, “Manage risk with the goal of implementing all the controls on the NIST Cybersecurity Framework.” FAIR is a standard that says, “Manage risk by analyzing loss event scenarios based on a set of shared terminology and processes.”
As Kahneman writes, “Rules simplify life, and reduce noise. But standards allow people to adjust to the particulars of the situations. Rules or standards? First, ask which produces more mistakes.”
The book is Noise: A Flaw in Human Judgment by Daniel Kahneman, Olivier Sibony and Cass R. Sunstein..
