Prepare to have at least some of your preconceptions about risk, cyber and otherwise, blown away by Douglas W. Hubbard in this video of his talk at the 2019 FAIR Conference, “How to Measure Risk with Limited and Messy Data: Overcoming the Myths.” Doug, the author of How to Measure Anything, How to Measure Anything in Cybersecurity Risk and The Failure of Risk Management, was one of the inspirations for Jack Jones’ creation of FAIR, in particular the revolutionary idea that cyber risk is NOT a special snowflake and can in fact be measured with statistical rigor.
Watch the complete video of Doug Hubbard’s talk How to Measure Risk with Limited and Messy Data: Overcoming the Myths. Slides are attached.
Note: FAIR Institute membership and LINK community site membership required. Join the FAIR Institute now (it's free).
To listen to Doug is to hear one "aha" moment after another that together will send you off more confident than ever about FAIR methods, and the wisdom of challenging conventional risk measurement. Here’s a sampling:
- “How many of you have heard this: We’d like to measure that but we don’t have sufficient data. They’re winging it. They don’t really know that…How many of you have heard the phrase ‘statistically significant sample size’? There’s no such thing…By that I mean there is no universal number where, if you are one short of it, you can’t make an inference.”
- If you plot a function that plots the expected cost of information and the expected value of information, the high payoff measurements tend to be relatively early…”The more uncertainty you have the more uncertainty reduction you get from the first few observations…The belief that we have more uncertainty so we need more data – mathematically just the opposite is true.”
- “Reputation damage seems like a hard thing to measure, doesn’t it?..But this is one area where you have all the data…There is no such thing as secret damage to reputation.” Looking to stock price as an indicator of reputation damage is a dead-end: studies have found that half the time prices go up after a cyber event and half the time they go down. The real cost is in “penance projects,” upgrading your security operation for instance – so what you are really forecasting in reputation loss is the cost you incur to head off future big losses.
- “Some subjective estimation methods measurably outperform others.” Subject matter experts can be trained in basic inference methods such as calibrated estimation, and avoiding common errors and biases. “With training, 85% of them are statistically indistinguishable from bookies. Bookies are very good at this. Physicians are terrible at estimating probability.”
Watch a teaser trailer of the session below:
Watch the complete video of Doug Hubbard’s talk How to Measure Risk with Limited and Messy Data: Overcoming the Myths. Slides are attached.
Note: FAIR Institute membership and LINK community site membership required. Join the FAIR Institute now (it's free).
Related:
Watch the Video from FAIRCON19: Perfecting a CISO Board Presentation with James Lam and Chris Inglis
