If you’re looking to understand the problem space for generative AI risk management and take first steps to apply FAIR quantitative risk analysis to GenAI, this video session from the recent FAIR Conference (FAIRCON23) is the place to start.
Presenters:
Brandon Sloane, Information Security Risk Management Lead, AI GRC, Meta
Pankaj Goyal, Director, Standards & Research, FAIR Institute
A free FAIR Institute membership required to view. Join now.
A few of the many takeaways on AI risk from this session:
Brandon: “A great way to think about this is the same way we approached cloud years ago…Think about what it means when you leverage somebody else’s service. What are you giving up and what are you gaining?”
>>Scenario 1 – Disclosure of private, confidential or sensitive information
They focused here on estimating the Susceptibility factor in FAIR modeling, dependent on strength of controls. Pankaj demonstrated research work done by Safe Security and the FAIR Institute and shown on GenAIRisk.ai: a library of controls relevant to AI and mapped to the FAIR Controls Analytics Model (FAIR-CAM™), a scoring of vendors of third-party SaaS services for AI based on their security features, and a library of GenAI risk scenarios.
>>Scenario 2 – Regulatory or user harms due to generation of undesirable content
They discussed the challenge of defining Loss Event Frequency in FAIR analysis. “The way we have defined ‘loss event’, it’s been very clear that a loss event has occurred,” said Brandon. But take the example of an AI generating a joke that’s offensive by some definitions: “Is this a loss event to every person who sees it?...This is an open question for the FAIR community.”
FAIR quantitative risk analysis could help at each phase of development of an AI product. Pankaj and Brandon outlined the process, starting from
But “nothing different here – we’ve been doing this for years,” Brandon said. “The only challenge is, what are the AI risks that you haven’t had to deal with before and are there additional controls and costs for standing these things up to bring us to an acceptable risk?
Watch the Video on Demand Now: Accelerating Your GenAI Adoption through AI Risk Posture Management
A free FAIR Institute membership required to view. Join now.
Also from FAIRCON23:
Good News on AI Risk: We Can Analyze It with FAIR