>>DHS/OMB mean well in pushing for a risk-based approach to cybersecurity in the Federal Government, but their requirements fall short of helping agencies effectively prioritize and right-size their cybersecurity investments
>>Agencies continue to focus mostly on meeting technical compliance requirements, as there aren’t clear and measurable expectations for assessing cybersecurity risk and effectiveness of controls
>>The good news is that a standard risk taxonomy and risk quantification methodology such as FAIR has emerged; one that complements the NIST CSF and NISTIR 8170 and 8286 standards and that enables cost-effective cyber risk management (the ultimate goal of the NIST CSF)
>>The private sector has been adopting FAIR for years now and the first federal agencies started implementing it in conjunction with the NIST CSF to better prioritize cybersecurity initiatives and right-size them based on actual risk
Nick Sanna is President of the FAIR Institute
Cyber Risk Requirements in the Federal Government
As cyber threats continue to multiply, Federal Agencies have been struggling to effectively assess and manage cybersecurity risk, as required by the following federal laws, executive orders and policies designed to help protect federal systems:
FISMA: The Federal Information Security Modernization Act of 2014, related executive orders and guidance from the Office of Management and Budget (OMB) and the Department of Homeland Security (DHS) explicitly require risk-based strategies and management processes for information security.
Executive Order 13800 (EO): The EO issued in May 2017 holds agency heads accountable for implementing risk management measures commensurate with the probability and magnitude of the harm resulting from cybersecurity events.
NIST CSF: The NIST Cybersecurity Framework provides a set of best practices for managing cybersecurity risk that the EO requires all federal agencies to adopt and that include risk assessments that can identify the potential business impact and likelihoods.
NIST Special Publication 800-39 states that cyber risk management strategies should include (1) a statement of the agency’s risk tolerance, (2) how it intends to assess risk (e.g., acceptable risk assessment methodologies), (3) acceptable risk response strategies (e.g., acceptance, mitigation, avoidance), and (4) how the agency intends to monitor the risk over time.
NIST SP 800-37 list the following elements to be included in risk-based policies, among other things: requiring an agency-wide assessment of cyber risks, developing an agency-wide strategy for monitoring control effectiveness, requiring system-level risk assessments to be performed regularly, tailoring system controls based on risk, prioritizing remedial actions based on risk.
According to both NIST 800-39 and 800-37, agencies should assess cybersecurity and privacy risks regularly, both at a system and at an aggregate agency level to identify systemic weaknesses and deficiencies in their systems and their impact on operations, assets and individuals.
OMB Circular A-123 directs agencies to implement an enterprise risk management (ERM) strategy and establish coordination between cybersecurity and ERM. OMB sees an effective enterprise risk management program as one promoting common descriptions of potential impacts on an agency’s mission.
NIST SP 800-39 states that effective risk management requires an agency’s mission/business processes to explicitly account for information security risk when making operational decisions and that cybersecurity risk information should be shared with key stakeholders throughout the organization.
GAO and FAIR Institute Findings
Following an audit of actual practices (GAO-19-384), the GAO states that Federal agencies have not fully established elements of their cybersecurity risk management programs. While most agencies have established the role of a cybersecurity executive and implemented policies where activities should be guided by risk-based decisions, fewer than half developed an agency-wide cyber risk management strategy and tied it to a wider enterprise risk management program.
The FAIR Institute’s own findings, in conversation with representatives from OMB, DHS, and several federal agencies suggest that the progress on implementing effective cyber risk management strategies is even more tenuous, as most cybersecurity programs are based on technical compliance versus a risk-based approach. The two are not mutually exclusive and should complement one another as, in a world of finite resources, technical cybersecurity investments must be prioritized and right-sized according to each agency’s threat activity, risk exposure and risk appetite.
OMB/DHS Guidance Is More Prescriptive on Technical Compliance than on Risk Management
While OMB and DHS took the great step to align government-wide cybersecurity guidance with the best practices outlined in the NIST CSF through OMB’s Memorandum M-20-04, the CIO FISMA Metrics, and the IG FISMA Reporting Metrics documents, no clear and prescriptive guidance is provided in how to assess, manage and report cybersecurity risk.
NIST CSF has traditionally referred to qualitative methods for risk assessment and provided only generalized guidance for developing a risk management program, which lacked the formal, specific, and repeatable procedures that are necessary to achieve the stated goal of enabling “cost-effective decision making”.
Federal agencies have been left to define their risk assessment approaches on their own, which often leads to (1) uneven risk definitions and (2) qualitative risk analysis methods that are imprecise, hard to compare and aggregate and do not enable effective prioritization and resource allocation.
NIST Is Starting to Point to the Benefits of Complementing the CSF with Quantitative Methods for Risk Assessment
The emergence of an open standard risk taxonomy and risk quantification model such as Factor Analysis of Information Risk (FAIR) provides a complementary analytics layer that adds an actionable, economical dimension to the implementation of the NIST CSF. FAIR is designed to assess cybersecurity risk in financial terms, as a basis for effective prioritization of risk mitigations and for calculating cybersecurity return of investment.
NIST has formally published FAIR as an Informative Reference to the NIST CSF, to reflect the mapping between FAIR and the NIST CSF in the sections covering risk analysis (ID.RA) and risk management (ID.RM).
NISTIR 8286, the new standard for integrating Cybersecurity and ERM, lists FAIR among valid risk analysis methods and states that “the cybersecurity risk officer may benefit from considering a quantitative methodology, with a more scientific approach to estimating likelihood and impact of consequences (..) to better prioritize risks or to prepare more accurate risk exposure forecasts.”
Traditional Qualitative Risk Assessments Fall Short
Qualitative risk assessments most often do not rely on accurate and consistent definitions of risk, and measure it in terms of high-medium-low, red-yellow-green or based on ordinal scales such as 1-4. Risk management and decision making based on qualitative risk assessments are inherently flawed and inaccurate as they are subject, among other issues, to:
Subjectivity: “how much risk is a ‘medium?”, “Is your medium the same as someone else’s?”
Range compression: using discreet numbers can lead to two scenarios being scored the same way but representing wildly different amounts of risk.
Inability to express uncertainty: using discrete numbers also results in the failure to express the possible range of outcomes, which should be core to risk analysis.
The consequence of using qualitative risk assessments is that the wrong prioritization decisions might be made as they are based on inaccurate risk estimates and that investment decisions continue to be made on a highly subjective basis, with little quantifiable data and economical rationale.
Quantitative risk assessments that leverage models such as FAIR help organizations prioritize risks and risk mitigation investments by quantifying the impact a risk presents to an organization’s operations, assets, and mission. Data and estimates are used to inform risk factors such as event frequency (likelihood) and event magnitude (impact), to obtain a range of probable loss exposure.
Using FAIR as part of a NIST CSF program can allow agencies to answer questions such as:
>>How much risk do we have? What are our top risks?
>>Which NIST CSF activities are most effective in reducing risk to an acceptable level?
>>What is our actual risk appetite? (explicitly, in dollars and cents)
>>What is the cost-benefit of improving the score of certain NIST CSF activities?
Articulating cybersecurity risk in financial terms helps better meet the requirements of assessing the adequacy of cybersecurity investments in the face of the specific risks an agency faces and bridges the communication gap between the cybersecurity team, executive management (CIO, CFO, agency head), and the reporting, auditing and oversight functions (OMB, IGs, GAO, Congress).
Overcoming Challenges Related to the Adoption of Quantitative Risk Assessments in the Federal Government
The private sector has been transitioning to quantitative risk management practices in increasing numbers since the emergence of the FAIR standard. The FAIR Institute, formed in 2016, will see its membership surpass 10,000 members in 2020, with over 40% of the Fortune 1000 companies represented.
Adopting quantitative risk assessments in federal government agencies will require the development of proper skills and expertise, as many agencies do not have mature risk management practices in place and are not used to measuring the impact of cyber events in quantitative terms.
Training is an absolute must for successful adoption. Certification courses such as ‘FAIR Fundamentals’ provided by RiskLens (Technical Sponsor of the FAIR Institute) teach risk managers how to properly define and scope risk scenarios, how to analyze them, and how to leverage techniques such as calibrated estimates and Monte Carlo simulations that are foundational to quantitative risk assessments.
Critical thinking is another attribute that risk managers must possess. Translating the consequences of cyber events in certain cases may not look obvious at first, especially when talking about mission impact regarding national security or protection of critical infrastructure that hasn't been translated into financial terms before. In these cases, knowing how to break down problems into more discrete factors up to the point where they become measurable is a success factor.
Forums such as the FAIR Institute and its Federal Government Chapter can also be of help, as best practices are shared among private organizations and federal agencies and network opportunities exist to uncover talent, expertise, and solution providers.
Please Contact Us if you are interested in joining the Federal Government Chapter of the FAIR Institute or to be connected to federal agencies that are already building quantitative risk management programs with NIST CSF and FAIR.