Critical Thinking – it’s always promoted as a core skill needed for any Factor Analysis of Information Risk (FAIR™) practitioner. Rightly so. The beauty of the FAIR standard is that it provides a framework for critical thinking that enables the quantification of risk including cyber, technology, etc. risk. To practically apply the FAIR methodology, a critical thinking acumen ensures analyses optimize the value FAIR offers by capturing the right inputs which subsequently produce meaningful outputs.
Over my years as a FAIR practitioner, I’ve observed some outlandish (humorously so) estimates. Over time, I’ve come to the realization that while critical thinking is crucial, common sense is equally meaningful. Seems obvious and straightforward, right? But, no, no, no… believe it or not, it is not obvious to all.
Ironically, it’s surprising to see how uncommon common sense can be at times when quantitative analysis enthusiasts become enamored with “critical thinking.” Since I often find myself a fan of underdogs and underrepresented entities, I wanted to relay three of my favorite experiential observations where common sense grounded critical thinking in reality.
Learn more: Are You a "Play It Safe" or "Get It Right" Risk Analyst? Take the Test, Learn Your Habits
Just a Digit Off
A financial organization was conducting an analysis with the overarching theme of data loss prevention. This large-scale, complex analysis included over a dozen scenarios. To ensure consistency and efficiency, loss tables were used to help answer the magnitude side of the analysis.
I didn’t have visibility into the data filling the loss tables, but I did have visibility into the results. When reviewing the results, I noticed there was a glaring, disproportionate delta between some scenarios. From what I could tell, when the affected records hit a certain threshold, the $ results were exponentially higher.
The analyst, who had demonstrated strong critical thinking, was apprehensive about having me review his calculations as they were (obviously) on point. After convincing the analyst it would behoove him to show me the loss tables, a quick glance illuminated the source of the exponential leap.
One field had $999M instead of $99M. I pointed out that there was the typo that was driving the outputs. The analyst shrugged his shoulders and said, “It’s just a digit off.” A digit off?! It was a $900,000,000 digit off! (Please, can my salary be “a digit off” ?!)
Moral of the story: Typos happen when people are in crunch-time mode and moored down in critical thinking. Don’t blindly believe the “data” or “calculations.” Bringing common sense to the quality assurance phase of the analysis process helps ensure the outputs are reasonable.
Interested in FAIR analysis training? Contact the FAIR Institute’s FAIR Enablement Specialists at fes@fairinstitute.org
50 Billion People
A global organization was looking to assess the amount of risk associated with a breach of a personally identifiable information (PII) from a large database. To help answer the “how much” side of the analysis, estimates were gathered on the large volume of unique PII records within said database to serve as the basis for notification costs, credit monitoring expenses, etc. Being a large, global, company… there were very large amounts of records. Estimates were made with a capital B, not an M (B standing for billions of course).
When it came time to review the analysis outputs, I noticed something was wildly off because the results were forecasting several billions ($) worth of exposure in a disproportionate amount to the size of the company.
Learn more: Calibrated Estimation for FAIR™ Cyber Risk Quantitative Analysis - Explained in 3 to 4 Minutes
I asked the analysts to walk me through the inputs to see what was driving the numbers. The team showed how they estimated the population of records within the in-scope database (~ 500 billion), then used critical thinking to say up to 10% of the records were PII records.
No need for a calculator here! Mental (or back-of-the napkin) math would quickly show 500,000,000,000 x 10% = 50,000,000,000. That’s a phenomenally big number! In fact, it’s an unbelievably big number when compared to the world population…. Literally. Not. Believable. The current world population is < 8 billion. Nowhere near 50B. I haven’t tested my hypothesis, but I don’t believe 50B people have lived on the face of the earth (if we aggregated) since the beginning of time!
Moral of the story: Don’t be afraid to use quick mental math to run a reasonability check. And remember, the sum (of some) of the parts can’t be greater than the whole.
You Can’t Give (or Lose) What You Don’t Have
One last one for giggles and grins. An analytics company conducted a detailed quantitative analysis on a breach scenario. A large concern for the company was the potential impact to reputation (which is one of the six forms of loss captured in FAIR analyses).
A good deal of critical thinking went into estimating the monetary impact associated with customer churn. However, when it came time to reviewing results, it became apparent that the reputation loss was forecasting several billion dollars’ worth of impact… and what was rather problematic is that the amount was more than double the company’s entire annual revenue. Literally. Double. A commonsense observer politely pointed out that the company would cease to exist because it couldn’t lose what it doesn’t have (said morbidly, it can’t die twice).
Moral of the story: you can’t give (or lose) what you don’t have. If the calculations are forecasting several times your financial capacity for loss, then maybe you should revisit and refine assumptions.
In closing, I hope the humorous (but real) observations where common sense was incredibly helpful inspire people to celebrate this skill more often. Said differently, don’t think so critically that common sense goes by the wayside.
