Responsible AI
Third-Party, Vendor, Open-Source, and Supply-Chain Governance
Govern procurement, open-source adoption, subcontractors, foundation models, data suppliers, contracts, change, incidents, and exit.
By the end you can
- Explain why third-party AI governance retains buyer accountability through evidence rights, local controls, change management, incident cooperation, and viable exit
- Distinguish Vendor certification, Contractual warranty, and Buyer-side system governance
- Identify evidence that connects pre-contract diligence to exit and continuity
- Design a review that moves from map the dependency chain to reassess continuously
The clause the buyer never obtained
Outsourcing technology does not outsource the buyer's responsibility for its use. US bank regulators wrote that down together in 2023, in guidance that retired two older OCC bulletins. The introduction puts it plainly: "Importantly, the use of third parties does not diminish or remove banking organizations' responsibilities to ensure that activities are performed in a safe and sound manner and in compliance with applicable laws and regulations." Subcontractors sit inside that perimeter too.
So the buyer's own governance has to carry what the contract cannot move. Due diligence, evidence rights, use restrictions, change notification. Security and incident duties, monitoring, remedy, portability, exit. All of it settled before the dependency becomes irreversible.
Europe stopped hoping for that paperwork and wrote it into law. Under Article 25(4) of the EU AI Act, the provider of a high-risk system and every supplier of an integrated component, tool, service or process must agree in writing. The agreement has to cover the information, capabilities and technical access that compliance needs. Free and open-source components are carved out, unless they are general-purpose AI models.
The chain may run through data brokers, annotators, cloud providers, foundation-model vendors, open-source maintainers, integrators, evaluators and downstream distributors. Controls should follow what each party can actually do to change the risk. Not the label the contract gave it.
Change notification is the clause the fraud-scoring buyer never obtained. The vendor moved the model without one.
Contract leverage exists only before signature. Every term skipped there — change notification above all — is something the vendor is free to refuse later.
Case
Federal agencies had to ask what was inside
Software supply chains hit this problem first. They built a vocabulary for it, and the vocabulary is worth borrowing whole. Executive Order 14028, signed on 12 May 2021, directed federal agencies to require a software bill of materials. A list of the parts, from the people selling the assembly.
A buyer who cannot name the parts cannot govern them.
Case
Three formats, and the library nobody knew they had
NTIA published the minimum elements that July and named three accepted formats: SPDX, CycloneDX and SWID. Five months later, Log4Shell showed what the list is for. The Log4j2 flaw published in December 2021 scored a maximum 10.0. Most affected organisations held the library only as a transitive dependency. Nobody had chosen it. That is exactly the inventory question an SBOM exists to answer.
An AI system needs the same inventory for models, weights, data and evaluation results.
Contract labels do not tell you who can actually change the risk.
Comparison
Vendor certification, Contractual warranty, or Buyer-side system governance?
Certificates report on a scope. Warranties allocate a loss. Neither one tests the system you actually run. And the EU AI Act adds one more turn. A deployer that substantially modifies a high-risk system already on the market stops being a deployer. It becomes the provider.
Vendor certification
Provides evidence against a defined standard or control set.
- Can reduce duplicate diligence
- Scope and period may be narrow
- Does not prove local suitability
- Needs interpretation and exceptions
Contractual warranty
Allocates obligations and remedies.
- Creates enforceable duties
- Cannot make an unavailable control effective
- May exclude affected-person repair
- Needs operational monitoring
Buyer-side system governance
Tests and controls the actual deployment.
- Accounts for local data and workflow
- Retains independent stop authority
- Requires internal capability
- Cannot depend entirely on vendor claims
Visual
Diligence, contract, integration, and a way out
Diligence before the contract. Allocation inside it. Controls around the integration, assurance while it runs, and a way out that works.
Europe put the last one in law. DORA has applied since 17 January 2025, with no transitional period. It requires exit strategies for ICT services behind critical or important functions. Firms must be able to leave without disruption to business activities, loss of regulatory compliance, or detriment to service continuity. DORA also requires a maintained register of every ICT third-party arrangement. And it names what those contracts must contain: whether subcontracting is permitted, advance notice before processing moves, data returned on discontinuation, incident assistance at no additional cost, termination rights.
An exit that has never been tested is a plan, not a way out.
- 1
Pre-contract diligence
Purpose fit, evidence, rights, provenance, security, limitations, and supplier maturity.
- 2
Contract and allocation
Duties, audit access, notification, incident support, remedy, retention, and exit.
- 3
Integration controls
Local evaluation, wrappers, thresholds, authorization, monitoring, and fallback.
- 4
Ongoing assurance
Change review, incidents, performance, vulnerabilities, subcontractors, and attestations.
- 5
Exit and continuity
Portability, replacement, evidence preservation, revocation, and service continuity.
Steps
Two levels below the name on the invoice
Map the dependency chain first. The components that matter most usually sit two levels below the name on the invoice.
Census II measured how far down that goes. In March 2022 the Linux Foundation and Harvard researchers pooled more than half a million observations of open-source use in production applications. In one dataset, 136 developers had written more than 80% of the lines added to the top 50 packages.
Your supplier list stops at companies. Your risk does not.
1. Map the dependency chain
Identify components, data, subprocessors, maintainers, and decision influence.
2. Define evidence and rights
Specify local evaluation, audit, documentation, notification, and incident access.
3. Contract for operation and exit
Cover change, retention, remedy, portability, discontinuation, and cooperation.
4. Control the integration
Apply local thresholds, least privilege, monitoring, fallback, and requalification.
5. Reassess continuously
Track versions, vulnerabilities, incidents, subcontractors, performance, and concentration risk.
Key idea
Liability moved, the harm stayed
A contractual transfer of liability does not prevent harm, restore trust, or satisfy every public duty. And certificates find less than buyers hope. A long-trusted co-maintainer planted a backdoor in the xz-utils release tarballs 5.6.0 and 5.6.1. It scored a maximum 10.0 when it was published to the NVD in March 2024. No audit caught it. Andres Freund did, after ssh logins on Debian sid started burning unusual CPU. It had reached sshd only indirectly, through liblzma.
So do not accept "proprietary" as a complete answer when the evidence is what makes the use lawful or safe. Small suppliers and open-source projects may genuinely lack assurance resources. Dominant vendors may simply refuse to negotiate. Either way the buyer still has choices: adjust the use, isolate the risk, contribute the fix, pick an alternative, or decline to adopt. Silently accepting the evidence gap is the one option that is not governance.
The fraud-scoring vendor changed the model without notice, then answered the subgroup question with a generic certificate. That is the moment the buyer's evidence rights were supposed to exist.
When a vendor will not close an evidence gap, narrowing the use, isolating it, or declining to adopt remain the buyer’s answers.
Example
The supplier graph, three levels down
Behind the vendor is another vendor. This drill maps the whole chain before anyone signs.
- Supplier graph: Trace the vendor, its data sources, model providers, cloud, subcontractors, and critical maintainers.
- Evidence minimum: List the evidence without which the intended use cannot be responsibly approved.
- Change simulation: Assume the vendor changes model behavior tomorrow and test notification, rollback, and revalidation.
- Exit rehearsal: Estimate time, data, interfaces, history, skills, and service continuity required to leave.
Example
A fraud API that changed without notice
Vendor changes reach production faster than buyer governance does. At 04:09 UTC on 19 July 2024, CrowdStrike shipped a content update that crashed Windows machines running its Falcon sensor 7.11 and above. The fix went out at 05:27, 78 minutes later. Microsoft still estimated that 8.5 million Windows devices were hit — under one percent of all Windows machines, and enough to ground airlines. No customer had approved that change. No customer could stop it.
Now the smaller version. A bank buys a fraud-scoring API. The vendor uses a subcontracted feature provider, changes the model without notice, withholds subgroup data as proprietary, and limits audit rights to a generic certification.
- Dependency opacity: Critical data and model components sit behind several organizations.
- Evidence asymmetry: The buyer cannot assess local population behavior or failure slices.
- Change control: The vendor can update the service without buyer requalification.
- Contract gap: Service-level terms cover uptime but not decision quality, incidents, or remedy.
- Exit risk: Historic versions, decision traces, and migration support are not guaranteed.
The operational conclusion — third-party AI governance
Accountability does not transfer with the component. The evidence the buyer holds is the part that has to stay current.
Name the vendor change, incident, or concentration finding that would force the owner to redesign, restrict, remedy, or retire the system.
Key takeaways
- Responsibility for the use of a third-party AI system stays with the organization deploying it.
- The chain may include data, annotation, cloud, model, integration, and evaluation providers.
- Certifications and warranties are bounded evidence, not proof of local suitability.
- Contracts should cover change, incidents, evidence, retention, remedy, portability, and exit.
- Local evaluation, authorization, monitoring, fallback, and stop authority remain the buyer's job.
- Concentration and exit risk have to be governed before the dependency becomes irreversible.