In a perfect world, a quantitative cyber risk analysis would always leverage data that is both accurate and precise. Heck, every sort of financial analysis, whether personal or organizational, would leverage data and produce results that are both accurate and precise. Then, everyone would be rich and prosperous because of smart investments! Sadly, we do not live in a perfect world.
Life, in our imperfect world, is full of compromises. However, compromises aren’t always undesirable; they are often the basis of win-win situations. In the realm of ranges and data collection in risk analysis, a win-win situation comes when there is the proper balance (compromise) between accuracy and precision.
To understand that balance, it is important to first understand the difference between accuracy and precision. Hopefully a few absurd storytelling examples might help.
Video: What Is Risk? The Bald Tire Scenario
Accuracy and Precision in Football
*Apologies in advance for any Vikings or Seahawks fans; but Blair Walsh’s perennial proclivity for kicking wide left is a perfect example of how precision is less desirable than accuracy. *
Apologies aside: Blair Walsh, a professional kicker for the NFL, infamously and exactly hooked three field goals to the left of the golden uprights during a pivotal game. If his fans were lovers of precision, they would have applauded the precision of his kicks. However, his teammates, coaches, and fans all would have preferred “accuracy” because they would have scored and won the game. The same is true for decision makers, boards of directors, and shareholders: Analyses that provide the accurate potential loss exposure will help set decision makers up for winning decisions. Analyses that are precise, but precisely wrong, don’t help inform decisions.
So, to help differentiate the concepts, think of precision as “exactness” and accuracy as “correctness” … or “winning.” Accuracy is king.
The Goldilocks Moment
Remember that likable kids story of Goldilocks and the Three Bears? She kept coming across a range of options that had a spectrum of something that was “too” –e.g., hot or cold—on either end. The Goldilocks moment would come when she found something that contained a balance between the two contrasting concepts, something that was “just right.” Let’s walk through a modern Goldilocks story, risk analysis style.
Once upon a time… Goldilocks asked her friend, Alex the analyst alpaca, to estimate the amount of time it would take the three bears to return from their evening stroll. She asked because she was trying to decide if she had enough time to snag some porridge from the bears’ home. Alex, being a smart aleck, answered: “10 seconds to 10 hours.”
Goldilocks, frustrated that the range was useless because it did not help her make a decision, said: “Don’t fixate only on being 'too' accurate. Try again.” Alex, without any rigmarole, said, “2-2.5 minutes.” Here, Goldilocks frowned because she was skeptical of his precise answer … and the amount of rigor, or lack thereof, that formed it. Here, Alex’s range was, “too” precise.
Eventually Alex, recognizing that his friend needed his help to make a decision, recalled calibration. He walked through the calibration process (starting with the absurd, decomposing the problem, referencing what he knows etc.) and told Goldilocks that the bears take 15-25 minutes on their evening walk. Here, Goldilocks was happy because this range is “just right.” Why? Because it was accurate, with a useful degree of precision. This is the Goldilocks Moment. (So, FYI, in this rendition of the story, the bears returned after 22 minutes).
They all lived happily ever after. The end.
Keep in mind, the Goldilocks balance will depend on a variety of factors, including the purpose of the risk analysis and the resources (e.g. data, time etc.) available. Don’t forget the law of diminishing returns.
Be like Goldilocks. Strive for the winning compromise between accuracy and precision in the data collection phase the risk analysis process.
Related:
