What is an “Intelligent adversary?”
Just so that we’re all on the same page, let’s identify a few key characteristics of “intelligent adversaries:”
The underlying premise is that because we face intelligent adversaries in cyber risk scenarios, quantifying the risk associated with these scenarios is unreliable and a waste of time. Any estimates of risk will be inaccurate either because the bad actor(s) inherently change their methods/focus periodically, or because they anticipate our actions and adapt. Instead, the argument goes, we should stick to following well-established control best practices and/or limit our risk analyses to qualitative methods.
Broken logic and a failure to understand quantitative methods
There are several fairly obvious logical problems with the arguments against quantitative risk measurement and the suggested alternatives to quantitative analysis:
I suppose the grossly imprecise nature of High/Medium/Low qualitative risk ratings does appear to make them insensitive to changes in the landscape. What many people fail to recognize though, is that quantitative estimates expressed as ranges or distributions (which is how quantitative risk values should always be expressed) can be as imprecise as is necessary to faithfully reflect the dynamic nature of the risk landscape. For example: not at all sure about how frequently bad actors will attack your website? No problem — use a wider, flatter distribution to reflect that uncertainty. Oh, and by the way, if the landscape is so profoundly dynamic that you don’t think your uncertainty can be reflected quantitatively, then qualitative estimates aren’t going to be any better. In fact, qualitative ratings have additional limitations because it’s difficult to reflect potential variance across qualitative boundaries, and you generally never know whether the qualitative rating reflects best-case, worst-case, or something in-between.
The bottom line
For the life of me, I can’t figure out a logical argument for how the characteristics of intelligent adversaries are relevant to quantitative risk measurement but not to qualitative ratings, or that they somehow invalidate quantitative methods. The bad actors will still:
What is true, however, is that because intelligent adversaries add to the dynamic nature of the risk landscape, organizations need to update their analyses periodically. Of course, this is true for both qualitative and quantitative analyses. And despite common misperceptions, updating quantitative analyses is no more difficult than competently updating qualitative ones.
It seems clear to me that people who believe quantitative risk measurement isn’t viable because of “intelligent adversaries” either haven’t thought it through and/or they aren’t familiar with well-established quantitative methods like Monte Carlo, calibrated estimations, etc. Hopefully, this blog post clarifies things a bit.