Lineage, Provenance, and Release Evidence

A signed attestation can show a lot. It can show that a model artifact was built by an approved pipeline, from specific inputs. It cannot show that the labels represented the intended outcome. It cannot show that the evaluation population was appropriate, or that the model is safe for a new use.

Operational evidence needs several complementary records. Lineage explains relationships. Provenance describes production. Evaluation measures behavior, and approvals record accountable decisions.

Regulation now reaches the capability itself. Article 12(1) of the EU AI Act is a single sentence: “High-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system”. Article 12(2) ties that logging to identifying risks under Article 79(1), to the post-market monitoring of Article 72, and to deployer monitoring under Article 26(5). The duty lands on the design, not on any particular stored file. A system that cannot produce the record has already failed the review.

Each layer answers a different investigation question, and the separation is not a local convention. Annex IV of the EU AI Act enumerates the technical documentation a high-risk system must carry under Article 11(1). It draws the same lines this diagram does.

Point 2(d) demands datasheets covering the training data sets, their provenance and the labelling procedures. That is the data lineage layer. Point 2(g) demands the validation and testing procedures and the metrics used, including potentially discriminatory impacts — the behavioural layer. The same point then demands “test logs and all test reports dated and signed by the responsible persons, including with regard to pre-determined changes”. Dated and signed by named responsible persons is the decision-record layer, written into a statute rather than a team wiki.

A single undifferentiated “audit folder” cannot satisfy three obligations that the annex itself keeps apart.

Using one word for every traceability record hides gaps. The XZ Utils backdoor shows exactly which gap. Andres Freund reported it to the oss-security mailing list on 29 March 2024, and Red Hat rated it CVSS v3 base score 10.0.

The malicious code was never in the project's public Git repository. Red Hat Product Security's record for CVE-2024-3094 states it directly: “The malicious injection present in the xz versions 5.6.0 and 5.6.1 libraries is only included in the tarball download package. The Git distribution lacks the M4 macro that triggers the build of the malicious code.”

The malicious object was assembled during the release and build step. A lineage graph pointing at the clean upstream repository would have been accurate about the source and wrong about the artifact that shipped. That is why the left column and the middle column below can never be collapsed into one.

A commercial risk-prediction algorithm was sending extra care to the wrong patients, and it was doing it nationally. Obermeyer and colleagues published the analysis in Science on 25 October 2019. The system was live, commercial and nationally deployed. The authors describe it as one of the largest and most typical examples of a class of commercial risk-prediction tools. By industry estimates, tools in that class are applied to roughly 200 million people in the United States each year.

It was biased for a reason no build attestation can see. It predicted health care costs rather than illness. The abstract puts a size on the effect: “Remedying this disparity would increase the percentage of Black patients receiving additional help from 17.7 to 46.5%.”

On the same day, New York's Department of Financial Services and Department of Health wrote jointly to UnitedHealth Group about the same product, Impact Pro. The letter came from Superintendent Linda A. Lacewell and Commissioner Howard A. Zucker.

A lineage event may state that a job consumed two datasets and produced a model. Build provenance may state which builder, source revision, parameters, and dependencies created an artifact. Neither statement substitutes for a release review.

The evidence bundle should preserve who asserted each fact, under which trust boundary, and at what time. A self-reported field from an untrusted build step is not equivalent to metadata generated by a protected control plane.

SolarWinds Orion is the case that makes the distinction expensive. A threat actor injected malicious code in February 2020 into a file later included in Orion software updates. The updates were signed and vendor-issued, and they carried the attacker's code anyway. CISA issued Emergency Directive 21-01 on 13 December 2020, covering versions 2019.4 through 2020.2.1 HF1. The U.S. Government Accountability Office later put numbers on the spread: “SolarWinds estimated that nearly 18,000 of its worldwide customers could have received a compromised software update. However, SolarWinds further estimated that fewer than 100 customers were actually compromised by the threat actor.” Nine federal agencies were breached. Metadata existed at every step. What failed was the authority of the platform that produced it.

SLSA grades exactly that authority. Version 1.0 of the specification, announced by the OpenSSF on 19 April 2023, defines a build track in three levels. Build L1 (“Provenance exists”) means the provenance is present, and the specification says such provenance “is trivial to bypass or forge”. Build L2 (“Hosted build platform”) requires a hosted build platform that generates and signs it. Build L3 (“Hardened builds”) requires strong tamper protection, so that forging provenance “requires exploiting a vulnerability that is beyond the capabilities of most adversaries”. What rises across the three levels is not the amount of provenance. It is the cost of faking it.

The bundle should allow a reviewer to reproduce the release decision, not merely locate files.

Steps 1 and 3 are not house advice. NIST states them as an auditable task in the Secure Software Development Framework, version 1.1, published as Special Publication 800-218 in February 2022. Task PS.3.2 reads: “Collect, safeguard, maintain, and share provenance data for all components of each software release (e.g., in a software bill of materials [SBOM]).” The publication's own mapping ties that task to Executive Order 14028 of 12 May 2021, at §4(e)(vi), (vii), (ix) and (x). Subsection (vii) is the SBOM clause: “providing a purchaser a Software Bill of Materials (SBOM) for each product directly or by publishing it on a public website”.

What moved since is the obligation around the task. CISA released the Secure Software Development Attestation Form on 11 March 2024, built on the SSDF practices and implementing OMB M-22-18 and M-23-16. OMB Memorandum M-26-05, of 23 January 2026, rescinded both memoranda. The common form is no longer mandatory, and agencies may choose to use it on a risk basis, with SP 800-218 still named as a reference. The obligation moved. The named task did not.

Step 5 has a legally defined shelf life. Article 19(1) of the AI Act provides that “the logs shall be kept for a period appropriate to the intended purpose of the high-risk AI system, of at least six months, unless provided otherwise in the applicable Union or national law, in particular in Union law on the protection of personal data”. Article 18(1) sets a second clock. The technical documentation, the quality-management-system documentation, notified-body decisions and the EU declaration of conformity must stay at the disposal of national competent authorities for ten years after the system is placed on the market. Six months and ten years are two different retention systems for one release. A bundle designed for neither will lose the first before an investigation reaches the second.

A good review ends by listing residual uncertainty.

Provenance cannot validate labels. No attestation about how a risk-prediction algorithm was built would have shown that its target was health care costs rather than illness, and that gap was worth the distance between 17.7% and 46.5%. Lineage cannot guarantee completeness. The XZ Utils graph pointed at a Git repository that lacked the M4 macro, while the tarball shipped the backdoor. Approval cannot turn an unsuitable system into a suitable one, and a signed, vendor-issued update could have reached nearly 18,000 SolarWinds customers.

The release bundle is defensible when its boundaries are as explicit as its contents. It also has to be legible at six months, and at ten years.