🚀 New Holistic AI Tracker 2.0 – Now featuring Atlas, a heat-mapped world view tracking AI legislation, incidents, and regulations in real time.
Register for free
Learn more about EU AI Act
Join Webinar: Bias Detection in Large Language Models - Techniques and Best Practices
Register Now
Learn more about EU AI Act

The NIST’s AI Risk Management Framework Playbook: A Deep Dive

Authored by
Published on
May 14, 2024
read time
0
min read
share this
The NIST’s AI Risk Management Framework Playbook: A Deep Dive

The Artificial Intelligence Risk Management Framework (AI RMF) Playbook serves as a practical companion to the AI RMF, offering actionable and adaptable guidance for organizations. In this blog post, we’ll give an in-depth overview of the Playbook, including the first steps organizations can take to implement it. For an introductory piece on the AI RMF, check out the Core Elements of the NIST AI Risk Management Framework.

Key Takeaways:

  • The Playbook offers voluntary suggestions – it is not a step-by-step checklist that organizations must closely follow in order to successfully use it.
  • The Playbook structures itself to align with the AI RMF. Each major function of the AI RMF – Govern, Map, Measure, and Manage – is accompanied by actionable guidance.

Govern

As with the AI RMF Core, the Playbook first addresses the Govern function, which broadly recommends that a “culture of risk management is cultivated and present” as the first step towards successful AI risk management. It is the bedrock function of the AI RMF Core, without which the other functions cannot succeed, and therefore actions related to it should come before Map, Measure, and Manage.

The Govern function generally offers two types of recommendations: those that address broad organizational practices and cultural norms and those specifically targeted the AI system. Each subfunction within Govern is accompanied by a set of suggested actions, along with recommended transparency and documentation practices. For instance, subfunction Govern 1.6 – which relates to the AI system– can be broken down as follows:

Guidance Summary Suggested Actions Transparency & Documentation
Govern 1.6 Mechanisms are in place to inventory AI systems and are resourced according to the organization’s risk priorities.
  • Establish policies that define the creation and maintenance of AI system inventories.
  • Establish polices that define model or system attributes to be inventoried (e.g. documentation, links to source code).
  • Document the individual or team responsible for managing the AI system inventory.
  • Review processes in place for data generation, acquisition/collection, storage, etc.

An AI system inventory will include system documentation, incident response plans, data dictionaries, links to implementation software or source code, names and contact information for relevant AI actors. It is an important practice for an organization to maintain because they provide a holistic view of organizational AI assets. AI inventories can also quickly answer important questions, such as when a given model was last updated.

Other subfunctions within the Govern section address broad personnel practices within an organization. For example, Govern 4.1 recommends:

Guidance Summary Suggested Actions Transparency & Documentation
Govern 4.1 Organizational policies and practices are in place to foster a critical thinking and safety-first mindset in the design, development, deployment, and uses of AI systems to minimize negative impacts.
  • Establish policies that require inclusion of oversight functions (legal, compliance, risk management) from the outset of the system design process.
  • Establish whistle-blower protections for insiders who report on perceived serious problems with AI systems.
  • Document the AI system’s development, testing methodology, metrics, and performance outcomes.
  • Review whether the organization’s information sharing practices are widely followed and transparent, so that past designs can be avoided.

NIST emphasizes that a culture of risk management is critical to effectively triaging most AI-related risks. In some industries, organization implement three or more ‘lines of defense’ in which separate teams are held accountable for different aspects of the system lifecycle, such as development, risk management, and auditing. This approach may be more difficult for smaller organizations, who can alternatively implement the ‘effective challenge’ which is a culture-based practice that encourages critical thinking and questioning of important design and implementation decisions by experts with authority and stature to make such changes.

NIST also recommends red teaming as another approach, which consists of adversarial testing of AI systems under stress conditions to seek out AI system failure modes or vulnerabilities. Typically, these consist of external expertise or personnel independent from the internal team.

Map

The Map function instructs AI RMF users to survey the context in which a given AI system is working and identify any potential context-related risks.

The Map function is geared towards aiding users in navigating contextual factors associated with AI systems, specifically enabling them to pinpoint risks and broader contextual elements. Recognizing the significance of context, it's crucial for Framework users to integrate diverse perspectives on the AI system. This includes input from internal teams, external collaborators, end users, and any potentially affected individuals, among others, to ensure a comprehensive understanding during this phase.

Like the Govern function, Map also considers the dual aspects of an AI system: it makes recommendations for the organization itself as well as specifically for the AI system. Map 1.1. addresses the organizational practices that help achieve the Map function:

Guidance Summary Suggested Actions Transparency & Documentation
Map 1.1 Intended purpose, potentially beneficial uses, context-specific laws, norms and expectation, and prospective settings in which the AI system will be deployed are understood and documented.
  • Examine how changes in system performance affect downstream events such as decision-making.
  • Determine and delineate the expected and acceptable AI system context of use, including social norms, impacted individuals, groups and communities, and operational environment.
  • Review whether a plan specifically addresses risks associated with acquisition, procurement of packaged software from vendors, cybersecurity controls, and computational infrastructure, and data among others.
  • Review which actors are responsible for the decisions of the AI and if they are aware of the limitations.

Map 1.1 is especially concerned with how and how an AI system is used (known as ‘context mapping’). For this subfunction, organizations should be cognizant of the specific set or types of users along with their expectations; potential positive and negative impacts of systems use to individuals, communities, organizations, society and the planet. NIST notes that even highly accurate and optimized systems can cause harm. As such, they recommend that discussion and consideration of non-AI or non-technology alternatives in some cases. In the Map 4 function, NIST makes recommendations regarding the AI system, encouraging that organizations map the risks and benefits for all components of the system. Within that function, Map 4.2 addresses third-party technologies:

Guidance Summary Suggested Actions Transparency & Documentation
Map 4.2 Internal risk controls for components of the AI system including third-party AI technologies are identified and documented.
  • Track third parties preventing or hampering risk-mapping as indications of increased risk.
  • Supply resources such as model documentation templates and software safelists to assist in third-party technology inventory and approval activities.
  • Determine if the AI system can be audited by independent third parties.
  • Review the mechanisms established to facilitate the AI system’s auditability.

Map 4.2 may be especially helpful for organizations with AI systems that use open-source or otherwise freely available, third-party technologies, which may have privacy, bias or security risks.

Measure

The Measure function involves assessing, analyzing, or tracking the risks first identified in the Govern and Map functions. It includes quantitative, qualitative, or mixed-method tools techniques, and methodologies to analyze and evaluate AI risk and their related impacts. The Map measure is critical to informing this function and the result will inform the Manage function. Map function 1.1 addresses these metrics:  

Guidance Summary Suggested Actions Transparency & Documentation
Measure 1.1 Internal risk controls for components of the AI system including third-party AI technologies are identified and documented.
  • Establish approaches for detecting, tracking and measuring known risks, errors, incidents or negative impacts.
  • Monitor AI system external inputs including training data, models developed for other contexts, system components reused from other contexts, and third-party tools and resources.
  • Determine the monitoring of appropriate performance metrics, such as accuracy, of the AI system after deployment.
  • Review the testing the organization has conducted on the AI system to identify errors and limitations.

NIST notes that AI technologies present new failure modes compared to traditional software systems due to their reliance on training data and methods which directly relate to data quality. The AI RMF consistently emphasizes the sociotechnical nature inherent to AI systems, meaning that risks often emerge from the interplay between the technical aspects of the system and who operates and the context in which is it operated.

Measure 2 outlines how AI systems are evaluated for trustworthy characteristics, and Measure 2.11 specifically addresses fairness and bias:

Guidance Summary Suggested Actions Transparency & Documentation
Measure 2.11 Fairness and bias – as identified in the Map function – is evaluated and results are documented.
  • Conduct fairness assessment to manage computational and statistical forms of bias which include the following steps, including the identifying the types of harms
  • Leverage impact assessment to identify and classify system impacts and harms to end users
  • Review the adversarial machine learning approaches considered or used for measuring bias (e.g. prompt engineering, adversarial modes)

Fairness includes concerns for equality and equity by addressing bias and discrimination, for example. NIST separates bias into three categories: systemic, computational and statistical, and human-cognitive:

  • Systemic bias: Present in datasets, organizational norms, practices and processes across the AI lifecycle, and the broader society that uses AI systems.
  • Computational and statistical bias: Present in AI datasets and algorithmic processes. Often stem from systemic efforts due to non-representative samples.
  • Human-cognitive biases: Relates to how an individual or group perceives AI systems to decide or fill in missing information, or how humans think about purposes and functions of a system.

Manage

Once organizations have gathered and measured all necessary information about an AI system, they can respond to identified risks. The Manage function, within the AI RMFC, advises users on how to prioritize and address these risks based on their projected impact. It offers detailed guidance on resource allocation for managing Mapped and Measured risks regularly, including any necessary domain expertise acquired during the Measure function. Additionally, it covers aspects such as communication and incident reporting to affected communities.

In the Manage function, all preceding functions converge. The contextual insights acquired during the Map phase are utilized to mitigate the likelihood of system failures and their consequences. The systematic documentation practices established in Govern, and utilized throughout Map and Measure, bolster AI risk management and enhance transparency during Manage. Just like the other functions, Framework users should continuously apply the Manage function as the AI system, the organization, and the contextual needs evolve over time. In Manage subfunction 1.2, the Playbook provides instructions for actions on an organizational scale:

Guidance Summary Suggested Actions Transparency & Documentation
Manage 1.2 Treatment of documented AI risks is prioritized based on impact, likelihood, or available resources or methods.
  • Assign risk management resources relative to established risk tolerance. AI systems with lower risk tolerances receive greater oversight, mitigation and management resources.
  • Document AI risk tolerance determination practices and resource decisions.
  • Review the organization’s risk management system to address risks involved in deploying the identified AI solution (e.g. personnel risk or changes to commercial objectives).
  • Review whether an organization has an existing governance structure that can be leveraged to oversee the organization’s use of AI.

NIST defines risk as the “composite measure of an event’s probability of occurring and the magnitude (or degree) of the consequences of the corresponding events.” It notes that the impacts, or consequences, of AI systems can be positive, negative, or both and can result in opportunities or risks. Organizational risk tolerance plays an important role in the Manage function as it determines how organizations choose to respond to the identified risks found in the Map function. Risk tolerance is typical informed by several internal and external factors, including existing industry practices, organizational values, and legal or regulatory requirements. In Manage 3.2, NIST addresses responses for AI models in particular:

Guidance Summary Suggested Actions Transparency & Documentation
Manage 3.2 Pre-trained models which are used for development are monitored as part of regular monitoring and maintenance.
  • Identify pre-trained models within AI system inventory for risk tracking.
  • DEstablish processes to independently and continually monitor performance and trustworthiness of pre-trained models, as part of third-party risk tracking
  • Review of how the entity has documented the AI system’s data provenance, including sources, origins, transformations, augmentations, labels, etc.
  • Review of how the organization ensures that the data collected is adequate, relevant, and not excessive to the intended purpose.

In AI development, transfer learning is common, where pre-trained models are adapted for related applications. Developers often utilize third-party pre-trained models for tasks like image classification, language prediction, and entity recognition due to limited resources. These models are trained on large datasets and require significant computational resources. However, their use can pose challenges in anticipating negative outcomes, especially without proper documentation or transparency tools, hindering root cause analyses during deployment.

Prioritize AI Governance

Although voluntary, implementing an AI risk management framework can increase trust and increase your ROI by ensuring your AI systems perform as expected. Holistic AI’s Governance Platform is a 360 solution for AI trust, risk, security, and compliance and can help you get ahead of evolving AI standards. Schedule a demo to find out how we can help you adopt AI with confidence.

DISCLAIMER: This blog article is for informational purposes only. This blog article is not intended to, and does not, provide legal advice or a legal opinion. It is not a do-it-yourself guide to resolving legal issues or handling litigation. This blog article is not a substitute for experienced legal counsel and does not provide legal advice regarding any situation or employer.

Subscriber to our Newsletter
Join our mailing list to receive the latest news and updates.
We’re committed to your privacy. Holistic AI uses this information to contact you about relevant information, news, and services. You may unsubscribe at anytime. Privacy Policy.

See the industry-leading AI governance platform in action

Schedule a call with one of our experts

Get a demo