Skip to content
AI.info

Responsible AI

Algorithmic Impact Assessment and Proportionality

Design an impact assessment that frames purpose, alternatives, affected parties, harms, controls, residual risk, and approval conditions.

By the end you can

Comparison

Checklist completion, Impact assessment, or External audit?

A checklist records that a topic was mentioned. An impact assessment builds a decision record. An external audit tests whether either one was true. Two of those three columns can be filled in with measured numbers rather than adjectives.

The audit column has a hit rate. In 2022 the Netherlands Court of Audit examined nine algorithms running inside the Dutch government. It judged each from five perspectives: governance and accountability, model and data, privacy, IT general controls, and ethics. The framework was its own, built the year before. Here is what independent testing found, published on 18 May 2022: “The Netherlands Court of Audit found that 3 out of 9 algorithms it audited met all the basic requirements, the other 6 did not and exposed the government to various risks: from inadequate control over the algorithm’s performance and impact to bias, data leaks and unauthorised access.” That is what an audit is good for, and what it is not. A verdict on nine deployed systems, delivered after all nine were deployed.

The assessment column has a shape. Canada has run its algorithmic impact assessment as a questionnaire since 2019: 65 risk questions, 41 mitigation questions, sorting each system into impact Levels I-IV. Depth is set by the level rather than by the reviewer's appetite. Requirement 6.1.3 of the Directive on Automated Decision-Making then obliges review and update on a scheduled basis, and whenever the system's functionality or scope changes. That is the part a checklist has no way to carry.

FigureComparison · 3 columns

Checklist completion

Records whether topics were mentioned.

  • Efficient for low-risk screening
  • Can encourage box-ticking
  • May not change a decision
  • Weak evidence of control effectiveness

Impact assessment

Builds an evidence-based decision record.

  • Begins before commitments become irreversible
  • Includes alternatives and affected people
  • Links findings to conditions and owners
  • Requires reassessment after material change

External audit

Independently tests claims and controls.

  • Can add challenge and credibility
  • Usually occurs on defined scope
  • Does not replace management ownership
  • Useful for high-risk or regulated systems

Visual

The step after the decision

Screening, scoping, analysis, decision, follow-through. An assessment that ends at the decision leaves its own conditions unowned. Three of the instruments in this lesson write that last step into binding text rather than into guidance.

Canada's requirement 6.1.3 obliges review and update on a scheduled basis, and whenever the system's functionality or scope changes. Article 27(2) of the EU AI Act attaches the fundamental rights impact assessment to the first use of the system. It adds a duty to update it when any element is no longer up to date. OMB Memorandum M-25-21 requires reassessment scheduling and post-modification triggers to be documented before deployment.

The effect is the same in all three. The reopening date exists on the day the system is approved, not after the first incident.

FigureProcess · 5 steps
  1. 1

    Screening

    Determine whether the use requires a full assessment and which expertise is needed.

  2. 2

    Scoping

    Define purpose, system boundary, affected parties, and decisions under review.

  3. 3

    Analysis

    Evaluate benefits, harms, rights, evidence, alternatives, and controls.

  4. 4

    Decision

    Approve, restrict, pilot, redesign, procure differently, or reject.

  5. 5

    Follow-through

    Track conditions, incidents, changes, reassessment, and retirement.

Early enough means before production, not before launch

An algorithmic impact assessment is a structured decision process, conducted early enough to change purpose, design, procurement, controls, or deployment. Its value lies in the decisions and the evidence it produces, not in how completely a form was filled in.

Early enough is the demanding phrase there. The administrations that mean it fix the deadline against production, not against the launch communications. Canada requires the assessment completed, approved and published before the system goes into production. The Article 29 Working Party requires a DPIA carried out prior to the processing, and started as early as is practicable in the design of the operation — even where some processing operations are still unknown.

What a strong assessment covers is also no longer a matter of taste. OMB Memorandum M-25-21, issued on 3 April 2025, names seven things a US federal agency must document, at minimum. The intended purpose and expected benefit, measured against existing agency processes. Data and model fitness. The potential impacts of using or not using AI. Reassessment scheduling and post-modification triggers. Cost analysis. The results of review by an independent reviewer inside the agency who was not involved in development. And risk acceptance, supported by a signature from the individual accepting the risk.

The third item decides whether the rest is real work. The potential impacts of using or not using AI is a comparison, and a comparison is only real where it can still be lost. How deep the rest goes should stay proportional to the impact and the uncertainty. That is what Canada's Levels I-IV, driven by 65 risk questions and 41 mitigation questions, exist to decide.

Fix the assessment date against production and procurement, not against the launch date. Once the comparison between using and not using AI can no longer be lost, the depth of everything else is decoration.

Key idea

Where an algorithmic impact assessment control can still fail

A high impact score should not automatically mean “add more controls and proceed.” Some uses stay disproportionate because a less intrusive alternative exists, or because the underlying purpose is unacceptable. A court has said exactly that about a deployed national system.

SyRI was the Dutch welfare-fraud risk-indication system. On 5 February 2020 the District Court of The Hague struck down the legislation behind it. It failed the necessity, proportionality and subsidiarity test under Article 8(2) of the ECHR. Section 65 of the SUWI Act and Chapter 5a of the SUWI Decree were declared to have no binding effect. The court put the reason in one sentence: “The court has decided that the legislation does not strike a fair balance, as required under the ECHR, which would warrant a sufficiently justified violation of private life.” No additional control was allowed to rescue it. The instrument came off the statute book.

Impact assessments run on incomplete evidence and contested values. Whoever writes the record should separate the verified facts, the assumptions, the uncertainty, the dissent, and the decisions that must be validated later. And should keep the SyRI outcome on the menu. An assessment whose only available answer is more controls has already conceded the purpose question.

Watch for the assessment whose only available answer is “add controls and proceed.” Necessity, proportionality and subsidiarity are three tests a control register cannot pass on the system's behalf.

Case

GDPR Article 35(1) and the nine screening criteria

Two European instruments already require an assessment of this kind, and they are not the same one. The first is Article 35(1) of the GDPR: a data protection impact assessment is required where processing is likely to result in a high risk to the rights and freedoms of natural persons.

Likely to result in a high risk sounds like a judgement call. The Article 29 Working Party turned it into a test. Its DPIA guidelines, adopted on 4 April 2017 and re-adopted in revised form on 4 October 2017, set out nine screening criteria. The European Data Protection Board later endorsed them. The list opens with evaluation or scoring, and with automated decision-making that has legal or similar significant effect. Meet two of the nine and, in most cases, a DPIA is required.

The guidelines fix the timing as well. The DPIA is carried out prior to the processing, and started as early as is practicable in the design of the operation. That holds even where some processing operations are still unknown. They also close the participation loop that risk-scoring deployments most often leave open: under Article 35(9) the controller must seek the views of data subjects or their representatives where appropriate.

On whether the exercise is ever finished, the guidelines take one sentence: “Carrying out a DPIA is a continual process, not a one-time exercise.”

Case

AI Act Article 27 adds a second assessment, and says what goes in it

The AI Act adds a second assessment on top of the data protection one, and this one is about fundamental rights. Article 27 of Regulation (EU) 2024/1689 binds a defined set of deployers of certain high-risk systems: public bodies, private entities providing public services, and deployers of certain Annex III point 5 systems.

It does not merely require the assessment. It prescribes six mandatory contents. The processes. The period and frequency of use. The categories of persons affected. The specific risks of harm. The human oversight measures. And the measures to take if the risks materialise, including internal governance and complaint mechanisms.

The obligation attaches to the first use, under Article 27(2), with a duty to update when any element is no longer up to date. Article 27(3) and 27(5) require the deployer to notify the market surveillance authority of the results, using the AI Office template. So the output leaves the building.

Article 27(4) settles how the two instruments stack: “If any of the obligations laid down in this Article is already met through the data protection impact assessment conducted pursuant to Article 35 of Regulation (EU) 2016/679 or Article 27 of Directive (EU) 2016/680, the fundamental rights impact assessment referred to in paragraph 1 of this Article shall complement that data protection impact assessment.”

Complement, not replace. The proportionality question sits inside both. Neither answers it for you.

Position

An administration that fixed the deadline, and named who signs

The useful test of an assessment is not whether the form was completed. It is whether anything was still reversible on the day it was completed.

Canada wrote that test into a rule. The Treasury Board Directive on Automated Decision-Making came into force on 1 April 2019, with compliance required by 1 April 2020. Requirement 6.1.1 puts the assessment before production, and puts a named official behind it, the responsible assistant deputy minister: “Completing, approving and publishing the final results of an algorithmic impact assessment in an accessible format on the Open Government Portal prior to the production of any automated decision system.” A named signer, a fixed gate, and a published result that anyone can read.

Requirement 6.1.3 then obliges review and update on a scheduled basis, and whenever the system's functionality or scope changes. The questionnaire behind it — 65 risk questions, 41 mitigation questions — sorts the system into impact Levels I-IV. Depth follows the level rather than the schedule.

The European instruments are specific about timing where they speak at all. Article 35(1) of the GDPR requires the assessment prior to the processing, where a type of processing is likely to result in a high risk to the rights and freedoms of natural persons. The Working Party adds that it should be started as early as is practicable in the design of the operation. Article 27 of the AI Act binds bodies governed by public law, private entities providing public services, and deployers of certain Annex III point 5 systems. That is a defined set of deployers and a defined set of systems. Reading it as a duty on everyone would be reading it wrong.

What none of these articles supplies is the answer. All of them leave the proportionality question inside the assessment. So the thing worth checking is narrow. Was the scope, the vendor, or the decision to proceed at all still open on the day it was signed? An assessment that could not have changed any of those is a document about a decision that was made elsewhere.

A form completed after signature records the decision. It does not make it.

Example

A risk indicator no control register could have fixed

On 7 December 2021 the Dutch Data Protection Authority fined the Dutch Tax Administration EUR 2.75 million. The system at issue automatically designated childcare-benefit applications as risky. One of the indicators it used was the applicant's nationality.

The regulator did not treat that as a control gap, to be closed with better oversight, sharper thresholds or a fuller mitigation table. Its finding was flat: “It is unlawful, and therefore prohibited, to use nationality data to assess applications, combat fraud and determine risk.”

  • Purpose: applicants' nationality was used as an indicator in a system that automatically designated childcare-benefit applications as risky — an input, not an accident of the model.
  • Data: some 1.4 million people were still registered as dual nationals in the administration's systems in May 2018, although the data should have been deleted in January 2014.
  • Regulator's finding: using nationality data to assess applications, combat fraud and determine risk was held unlawful outright, and therefore prohibited. A purpose ruled out, not a risk scored and mitigated.
  • Consequence: a fine of EUR 2.75 million, imposed on 7 December 2021 for discriminatory and unlawful data processing.
  • Decision effect: no depth of assessment attached to this design could have produced a lawful approval, because the finding lands on the indicator itself. The only outcomes available were redesign or retirement.

Example

The last moment an assessment can change anything

Begin with the timing, and borrow the fixed points from administrations that already have them. Published before production, under Canada's requirement 6.1.1. Completed before deployment of any high-impact use case, under M-25-21. Carried out prior to the processing and started as early as is practicable, under the Article 29 Working Party's guidelines.

  • Decision point: name the last moment the assessment can still change scope or procurement, and write it as a gate rather than a target. Canada's gate is production, and the responsible assistant deputy minister signs it.
  • Alternative table: compare benefits, intrusiveness, cost, uncertainty and reversibility for at least three options. Keep the entry M-25-21 makes mandatory: the potential impacts of using or not using AI.
  • Condition register: turn each approval condition into a control, an owner, a deadline and a verification method. Add the two artefacts M-25-21 names: the results of review by an independent reviewer inside the agency who was not involved in development, and a signature from the individual accepting the risk.
  • Dissent appendix: preserve minority findings and unresolved evidence gaps, alongside the reassessment scheduling and post-modification triggers, so the questions left open have a date on which they are asked again.

Steps

Compare against no system at all

Screening belongs at the idea stage. As early as is practicable in the design of the operation, in the Article 29 Working Party's phrase, even where some processing operations are still unknown.

The counterfactual comes second, because the comparison against no system at all is the one most assessments leave out. M-25-21 makes it non-optional: agencies must document the potential impacts of using or not using AI, measured against existing agency processes.

Evidence comes third, and it should be plural. Technical tests, domain knowledge, legal analysis, incident history. Add the views of data subjects or their representatives, which Article 35(9) obliges the controller to seek where appropriate. The five perspectives the Netherlands Court of Audit used — governance and accountability, model and data, privacy, IT general controls, ethics — are a workable map of what an independent look actually covers.

Fourth, decide proportionately, and keep the full range open: approve, restrict, pilot, redesign, procure differently, or reject. The District Court of The Hague took the last of those with SyRI.

Fifth, reassess. On a scheduled basis and whenever functionality or scope changes, under Canada's requirement 6.1.3. And whenever any element of the Article 27 assessment is no longer up to date, under Article 27(2).

FigureProcess · 5 steps

  1. 1. Screen early

    Trigger review at idea, procurement, or material-change stage.

  2. 2. Define the counterfactual

    Compare AI, non-AI, and no-intervention alternatives.

  3. 3. Gather plural evidence

    Combine technical tests, domain knowledge, participation, legal analysis, and incident history.

  4. 4. Decide proportionately

    Approve, restrict, pilot, redesign, or reject with conditions and owners.

  5. 5. Reassess over time

    Reopen the assessment after changes, incidents, drift, or new evidence.

A condition without an owner is a wish

An assessment closes with conditions. A condition without a control, an owner and a date is a wish. So it is worth reading how a rule makes those three concrete.

M-25-21 states the gate in one line: “Agencies must complete an AI impact assessment before deploying any high-impact AI use case.” Around that sentence sit the artefacts that stop it being a slogan. The results of review by an independent reviewer inside the agency who was not involved in development. Risk acceptance supported by a signature from the individual accepting the risk. Reassessment scheduling and post-modification triggers. And 365 days for agencies to document implementation.

One clause gives the conditions teeth. A high-impact use case that is not compliant must be safely discontinued.

Define, in the same way and in advance, when your algorithmic impact assessment requires the team to redesign, restrict, remedy, or retire the system.

Key takeaways