FAIR standard creator Jack Jones spoke this week at the 2022 RSA Conference with the message that the future of risk measurement and management is (drum roll) artificial intelligence and automation. You might have heard the same in vendor booths on the show floor, but not like Jack told it: The industry won’t get there without a major shift left in current risk practices.
“Our profession has focused on fast and easy risk management without a clear understanding of what good measurement looks like or requires,” Jack said. Without good measurement, advancements in risk analytics like automation or machine learning will simply be faster routes to unreliable results.
According to Jack, cyber risk measurement of the future must…
1. Be quantitative
“Qualitative risk management has too many inherent limitations.” To take one egregious example, qualitative ratings dressed up as numbers (such as CVSS scores or NIST CSF maturity ratings) and performing math on them to achieve “risk quantification” that really isn’t.
2. Account for “controls physiology”
Like the study of physiology for the body’s organs, Jack’s new FAIR-CAM controls analytics model quantifies the effectiveness of cybersecurity controls and controls systems by carefully exposing the dependencies among them. “The controls landscape is complicated and highly nuanced,” Jack said, and without accounting for that, risk analysis, AI-powered or not, won’t be dependable.
Join the FAIR Institute, stay on top of the movement for better risk management through quantification.
3. Be based on open, non-proprietary models
A bold statement for a cybersecurity conference where so many black-box solutions are promoted. Jack’s point: “If we can’t examine it, we shouldn’t trust it.” FAIR is the only open, and independently vetted (by The Open Group) cyber risk quantification model in the industry.
4. Where possible, leverage automation and AI
Artificial Intelligence, Jack said, is particularly susceptible to weakness in data and risk models because it must be trained. Poor training data leads to inappropriate analysis conclusions, which become self-reinforcing, leading to bias that’s difficult to correct. What’s more, “the ability to audit and analyze how an AI arrived at a particular result is non-trivial. Unless the AI is auditable, it should be considered just as prone to failure as proprietary models.”
“FAIR-CAM is one of the keys to enabling automation as well as AI,” Jack added.
5
. Be performed by trained and certified professionals.
At too many organizations, Jack said, risk “analysis” is open to anyone who wants to sit at the table and pick a color: red, yellow or green. “Risk analysis and measurement should be considered a distinct discipline, just as forensics, penetration testing, DevSecOps and others are.” The FAIR Institute and the RiskLens Academy are doing their part:
>>Over 10,000 people have completed the FAIR Fundamentals training course.
>>Over 1,200 have passed the OpenFAIR certification exam.
Network and learn at the 2022 FAIR Conference, Sept. 27-28, Washington, DC
Also at the RSA Conference: Tips on FAIR Quantitative Risk Management Program Launch from FAIR Experts
Jack was joined at the RSAC22 session by two FAIR practitioners, Tony Martin-Vegue, Senior Information Security Risk Engineer at Netflix and Darren Kane, CSO at nbn, the Australian telecommunications company. Some of their tips:
>>Security people should get into the habit of talking about security “investments” that can be prioritized for cost vs. benefit and stop obsessing over risk mitigation. Tony said that “return on investment” is now his most requested analysis.
>>You’re more likely to get pushback from the IT folks – who are oriented toward frameworks compliance – and get welcomed by the business folks who are used to thinking in terms of loss exceedance curves, risk appetite and the other concepts that FAIR risk analysis shares with business risk analysis.
>>Take advantage of RiskLens or other software packages for quantitative risk analysis – do-it-yourself spreadsheets are a thing of the past.
>>Scoping is a critical skill to teach your staff. Focusing analysis in FAIR terms saves a lot of wasted effort on “risks” that may really just be concerns.
