For a fresh take on getting the most out of your risk analysts and subject matter experts (SMEs), watch the video of Douglas Hubbard’s session at the 2020 FAIR Conference: The Team as a Measurement Instrument. Hubbard’s message: Analysis is a team effort and you can significantly improve the quality of analysis by improving the quality of the team.
The business benefits are clear, not just for cybersecurity but for market expansion, M&A, product development or any new initiative: “Think of all the ways your organization relies on the estimates of subject matter experts and you have to ask: What is the performance of that method?'…Your biggest single risk is how you measure risk.”
Best known as the author of How to Measure Anything: Finding the Value of Intangibles in Business, one of the best-selling math books of all time, Hubbard has been a major influence on the FAIR movement, particularly with his advocacy for calibrated estimation as a foundational practice in data gathering for quantitative risk analysis.
Citing his research and other scientific studies on the decision process, Hubbard walks through the dynamics that make for teams that are best at data gathering and making reliable estimates. Some key characteristics of good players:
- Training in calibration and basic probability theory
- Aptitude, based on previous success at estimation
- ”Belief updaters” – open-minded individuals who can adapt to changing facts
Hubbard describes a system for aggregating SME estimates and performance weighting them (it turns out that, for a variety of reasons, a few experts tend to be superstar estimators, while most aren’t that effective) to create a final product that’s better than the sum of the parts – as he jokingly calls it, a “FrankenSME.” “This will probably be the next big thing after calibration, itself,” he concludes.
Watch the video of Douglas Hubbard’s FAIRCON2020 session The Team As a Measurement Instrument on the FAIR Institute’s LINK member resources site (Institute membership required – join now, then sign up for LINK access).
Hubbard also spoke at the 2019 FAIR Conference on How to Measure Risk with Limited and Messy Data.