“Our profession has focused on fast and easy risk management without a clear understanding of what good measurement looks like or requires,” Jack said. Without good measurement, advancements in risk analytics like automation or machine learning will simply be faster routes to unreliable results.
According to Jack, cyber risk measurement of the future must…
“Qualitative risk management has too many inherent limitations.” To take one egregious example, qualitative ratings dressed up as numbers (such as CVSS scores or NIST CSF maturity ratings) and performing math on them to achieve “risk quantification” that really isn’t.
Like the study of physiology for the body’s organs, Jack’s new FAIR-CAM controls analytics model quantifies the effectiveness of cybersecurity controls and controls systems by carefully exposing the dependencies among them. “The controls landscape is complicated and highly nuanced,” Jack said, and without accounting for that, risk analysis, AI-powered or not, won’t be dependable.
Join the FAIR Institute, stay on top of the movement for better risk management through quantification.
A bold statement for a cybersecurity conference where so many black-box solutions are promoted. Jack’s point: “If we can’t examine it, we shouldn’t trust it.” FAIR is the only open, and independently vetted (by The Open Group) cyber risk quantification model in the industry.
Artificial Intelligence, Jack said, is particularly susceptible to weakness in data and risk models because it must be trained. Poor training data leads to inappropriate analysis conclusions, which become self-reinforcing, leading to bias that’s difficult to correct. What’s more, “the ability to audit and analyze how an AI arrived at a particular result is non-trivial. Unless the AI is auditable, it should be considered just as prone to failure as proprietary models.”
“FAIR-CAM is one of the keys to enabling automation as well as AI,” Jack added.
At too many organizations, Jack said, risk “analysis” is open to anyone who wants to sit at the table and pick a color: red, yellow or green. “Risk analysis and measurement should be considered a distinct discipline, just as forensics, penetration testing, DevSecOps and others are.” The FAIR Institute and the RiskLens Academy are doing their part:
>>Over 10,000 people have completed the FAIR Fundamentals training course.
>>Over 1,200 have passed the OpenFAIR certification exam.
Network and learn at the 2022 FAIR Conference, Sept. 27-28, Washington, DC
>>Security people should get into the habit of talking about security “investments” that can be prioritized for cost vs. benefit and stop obsessing over risk mitigation. Tony said that “return on investment” is now his most requested analysis.
>>You’re more likely to get pushback from the IT folks – who are oriented toward frameworks compliance – and get welcomed by the business folks who are used to thinking in terms of loss exceedance curves, risk appetite and the other concepts that FAIR risk analysis shares with business risk analysis.
>>Take advantage of RiskLens or other software packages for quantitative risk analysis – do-it-yourself spreadsheets are a thing of the past.
>>Scoping is a critical skill to teach your staff. Focusing analysis in FAIR terms saves a lot of wasted effort on “risks” that may really just be concerns.