Skip to content
AI.info

Responsible AI

AI Management Systems: NIST AI RMF, ISO/IEC 42001, and Impact Assessment

Integrate NIST AI RMF, ISO/IEC 42001, impact assessment, risk management, internal audit, certification, and continuous improvement.

By the end you can

Comparison

NIST AI RMF, ISO/IEC 42001, or AI impact assessment?

Frameworks organize the work. Management-system standards make it auditable. An impact assessment examines one system in one context. The first two say nothing about whether one particular system, used in one particular place, is safe.

FigureComparison · 3 columns

NIST AI RMF

Voluntary risk-management framework organized around Govern, Map, Measure, Manage.

  • Flexible and outcome-oriented
  • Supports profiles and crosswalks
  • Not a certification standard by itself
  • Needs organizational tailoring

ISO/IEC 42001

Requirements for an AI management system.

  • Supports auditable management-system implementation
  • Can be used for certification within scope
  • Does not certify every product outcome
  • Integrates with other ISO systems

AI impact assessment

Examines a particular system, context, stakeholders, impacts, controls, and residual risk.

  • Produces use-specific evidence
  • Should occur before and after change
  • Cannot replace enterprise governance
  • Needs participation and remedy

Key idea

Clean audit, unusable appeal

A clean audit can sit alongside a bad product decision. That happens when the scope was narrow, the sampling thin, the evidence poor, or when nobody on the ground applied the process properly.

An assurance statement is supposed to name six things: the object, the criteria, the period, the methods, the findings, and the limitations. The object line settles the case above. The certificate covered a management system. The company announced a certified-safe product.

No framework resolves a value conflict, and none predicts every impact. That still takes people who judge well and stakeholders who actually take part. It takes lawyers who read the thing. Most of all it takes someone with the authority to reject a system while the management process around it looks mature. The product with no local subgroup evaluation and no workable appeal was inside what the company announced. It was outside what the certificate covered.

A clean finding reports on a sampled process, and the person left without a workable appeal never appears in the sample.

Visual

Where an AI management system enters the lifecycle

Govern, map, measure, manage, and the assurance loop that checks whether any of it operates. The last one is where a mature-looking program usually fails.

FigureProcess · 5 steps
  1. 1

    Govern

    Policies, roles, culture, literacy, accountability, inventory, and oversight.

  2. 2

    Map

    Context, stakeholders, intended use, impacts, laws, risk sources, and assumptions.

  3. 3

    Measure

    Tests, monitoring, uncertainty, fairness, robustness, privacy, security, and human factors.

  4. 4

    Manage

    Prioritization, treatment, acceptance, incident response, change, and retirement.

  5. 5

    Assure and improve

    Internal audit, management review, independent assessment, corrective action, and learning.

Repeatable is not the same as safe

Management frameworks give an organization repeatable ways to govern AI risk, assign ownership, keep evidence, audit controls, and improve. None of that guarantees that a given system is fair, safe, lawful, or effective in the place it is used.

Repeatability means the same decision would be reached again — a virtue when it was the right one, a liability when it was not.

Case

Only one of the three looks at one system in one context

Three documents sit behind most AI governance programs, and they do three different jobs. The NIST AI RMF organizes the work through Govern, Map, Measure, and Manage. ISO/IEC 42001 sets requirements for an AI management system. ISO/IEC 42005 gives guidance for assessing the impact of one AI system. Any of them can run alongside enterprise risk, quality, privacy, security, safety, and sector management systems.

Only the third examines one system, in one context. The other two describe how a company works. Nobody is harmed by a company in the abstract.

Predict first

Can you be certified against both?

Two of the AI documents ISO published in 2023 sit side by side in an AI management system. ISO/IEC 42001, in December, sets out a management system that auditors can certify. ISO/IEC 23894, in February, covers AI risk management.

Make a prediction before reading on. Is ISO/IEC 23894 a second certifiable standard, or a different kind of document?

Settle on an answer, then read on — the next section is the reveal

Case

Four documents, four dates, one certificate

The whole reference set arrived inside eighteen months. NIST released AI RMF 1.0 on 26 January 2023, built on four functions: Govern, Map, Measure and Manage. Its Generative AI Profile, AI 600-1, followed in July 2024 with twelve risk categories. ISO/IEC 42001 was published in December 2023 and can be certified against. ISO/IEC 23894, from February 2023, is guidance with no auditable requirements.

Four documents, and only one of them is auditable. None of the four looks at one system, in one context, with one set of affected people. Only the impact assessment does that.

FigureInteractive

A date axis from 2023 to 2025 places the four documents an AI management system answers to: NIST AI 100-1, the AI RMF 1.0, on 26 January 2023; ISO/IEC 23894, guidance, in February 2023; ISO/IEC 42001, the certifiable standard, in December 2023; and NIST AI 600-1, the Generative AI Profile, on 26 July 2024. The two NIST releases are eighteen calendar months apart to the day, and both ISO documents landed inside that window. Cards carry the counts: 4 AI RMF core functions, 12 generative-AI risk categories, and two ISO documents of which one is auditable and one is guidance. The ISO dates are given to the month in the record, so those marks are plotted mid-month.

The whole reference set an AI management system is audited against appeared inside eighteen months — and only one of the four carries auditable requirements.

Position

A certificate covers the object named on it

A certificate names an object. For ISO/IEC 42001, published in December 2023, that object is a management system. The standard is written as auditable requirements, and an organisation can be certified against them within a defined scope. That is what makes it useful. It is also what makes “certified safe” the wrong claim to build on it.

The auditor sampled roles, policies, records, audits and corrective action. Whether one product performs on the population it will be used on was not sampled, unless somebody separately looked.

NIST's AI Risk Management Framework is weaker still as an assurance claim, and says so itself. It was released on 26 January 2023 around Govern, Map, Measure and Manage. It is intended for voluntary use, and it is not by itself a certification standard. That has not stopped it appearing in assurance material as though naming it were a finding.

The reader's move costs nothing. Find out which object the claim names. Then read the scope for what it excludes. No standard requires a buyer to do that, which is precisely why so few do.

Establish what the certificate covers before asking whether it is valid.

Example

Certified as a company, sold as a product

A company earns certification for its AI management system. It then announces that every covered product is “certified safe.” One of those products still lacks local subgroup evaluation and a workable appeal process.

  • Management-system evidence: The organization has roles, policies, objectives, records, audits, and corrective-action processes.
  • Product evidence gap: A specific use lacks enough data about a consequential population.
  • Claim inflation: Certification scope is presented as outcome assurance for every product.
  • Audit boundary: Auditors sample evidence within a defined scope and period.
  • Continuous improvement: The system should detect and correct the gap rather than deny it.

Example

What the certificate does not cover

Read the scope statement for what it excludes. That is where the certificate stops covering anything.

  • Scope statement: Write what the management system includes and what it explicitly excludes.
  • Framework crosswalk: Map NIST functions to ISO processes and the evidence your organization actually keeps.
  • Certification-claim review: Rewrite a marketing claim so it accurately represents object, scope, and limitation.
  • Internal-audit sample: Select evidence that tests operation rather than document presence alone.

Steps

Turn the framework into an operating control

Scope, architecture, per-system assessment, audit, improvement. The audit step tests whether the controls actually operate, rather than whether the documents exist.

FigureProcess · 5 steps

  1. 1. Define the management scope

    Entities, systems, lifecycle, interfaces, suppliers, risks, and exclusions.

  2. 2. Build the control architecture

    Map policies, roles, processes, evidence, metrics, and escalation.

  3. 3. Conduct system impact assessments

    Apply context-specific stakeholder, harm, rights, and control analysis.

  4. 4. Audit claims and operation

    Sample evidence, observe workflows, test controls, and preserve findings.

  5. 5. Review and improve

    Correct causes, track actions, update objectives, and change risk decisions.

Example

The certified company, through the five steps

Once its AI management system is certified, a company announces that every covered product is 'certified safe', while one product still lacks local subgroup evaluation and a workable appeal. The quiz below is built on this invented case. Here it is run through the five steps. Every figure is invented for the exercise.

  • Define the management scope: the certificate names the AI management system of the product division at two sites. It names no product, so 'every covered product is certified safe' is rewritten as a statement that the management system is certified to ISO/IEC 42001, for that scope.
  • Build the control architecture: policies, roles and corrective action are in place and audited. What is missing is evidence about this product: a local subgroup evaluation and an appeal that works.
  • Conduct system impact assessments: the product will be used on 50,000 people a year, 6% of them over 75. The vendor's validation set held 120 people over 75; at 90% accuracy, that leaves a margin of error of about ±5 points, too wide to see a 5-point drop. Collecting 600 local cases brings it to about ±2.4.
  • Audit claims and operation: an internal auditor files a test appeal. It takes 41 days and never reaches anyone who can change the result. The finding goes into the corrective-action register with an owner.
  • Review and improve: management review sets a deadline for the appeal fix, stops marketing claims about the product until the local evaluation is done, and checks at the next audit that the corrective action worked.

Maturity is evidence about the organization

Process maturity is evidence about the organization, not about the system. That is why each system still has to be assessed on its own.

Name the audit finding, impact-assessment result, or overstated claim that would force the team to redesign, restrict, remedy, or retire the system.

Key takeaways