MLOps
Continuous Integration for ML Assets
Design CI that validates software, pipelines, model interfaces, policies, and evidence before expensive training or deployment begins.
By the end you can
- Expand continuous integration beyond application code to all behavior-changing ML assets
- Order CI stages from fast deterministic checks to selective expensive validation
- Prevent untrusted pull requests from accessing production secrets or infrastructure
- Define merge gates that produce reproducible evidence rather than ceremonial green checks
A one-line feature change can invalidate the whole system
A pull request changes a timestamp join from `<` to `<=`. Unit tests pass. Formatting passes, and the model code is untouched. The change introduces future information into training and silently improves offline metrics.
Three regulators have already written down what that one character breaks. In October 2021 the FDA, Health Canada and the UK's MHRA jointly identified ten Good Machine Learning Practice guiding principles for medical device development. The fourth is not advice about hygiene. It is a stated expectation: “Training Data Sets Are Independent of Test Sets: Training and test datasets are selected and maintained to be appropriately independent of one another. All potential sources of dependence, including patient, data acquisition, and site factors, are considered and addressed to assure independence.”
A join predicate is a source of dependence. It is also a two-character diff that no reviewer reads as one, and no unit test in the repository was written to see it.
ML CI must therefore know which assets can alter behavior: feature logic, label definitions, data contracts, schemas, dependencies, configurations, policies, prompts, and evaluation code.
Case
Leakage found in seventeen fields, sorted into eight types
Leakage is not one team's bad week. Two researchers went looking for it in the published record rather than in a pipeline, and in 2023 Kapoor and Narayanan reported in Patterns that “we find 17 fields where leakage has been found, collectively affecting 294 papers”, and that “we introduce a detailed taxonomy of eight types of leakage, ranging from textbook errors to open research problems”. Their earlier preprint counts higher — “we find 17 fields where errors have been found, collectively affecting 329 papers” — while the field count and the taxonomy size are identical in both versions.
Two numbers are worth holding together. Seventeen is the number of scientific fields, not the number of teams. Leakage is not a local lapse by careless engineers; it survived peer review across most of the empirical disciplines that use machine learning. Eight is the number of distinct mechanisms, and eight is the operationally useful figure. A list of eight named types is short enough to become a checklist and specific enough to become CI jobs. Some are textbook errors that a test can catch outright. Others are open research problems that a test can only surface for a human to argue about. A pipeline that has never been read against that list has not decided which of the eight it is exposed to. It has only never been told.
CI should fail early and spend evidence carefully
Fast checks should catch syntax, type, schema, policy, and deterministic transformation errors before anything expensive starts training. Costlier checks can then run only for changes that touch the relevant asset graph.
The system has to know what depends on what. A documentation change should not retrain a large model. A tokenizer or label change should trigger broader validation than an isolated UI update.
That idea has been made countable. In a 2017 paper on ML production readiness, Breck, Sculley and three colleagues “present a rubric as a set of 28 actionable tests, and offer a scoring system to measure how ready for production a given machine learning system is”. The 28 sit under four headings: “Tests for Features and Data”, “Tests for Model Development”, “Tests for ML Infrastructure” and “Monitoring Tests for ML”.
The shape of that list is the argument. Three of the four headings are about things that are not the model. A repository whose CI covers only model code is, by the rubric's own accounting, testing one quarter of the surface. This is a checklist and a readiness score, not a benchmark. Its value is that a team can walk it and find out which of the 28 it has never run.
Example
An action used by 23,000 repositories printed their secrets into public logs
On 14–15 March 2025 an attacker retroactively repointed the version tags of the third-party GitHub Action tj-actions/changed-files — v1 through v45.0.7 — at a malicious commit. Teams that had pinned a version tag, which is the recommended practice, got the new commit without changing a line of their own workflow.
StepSecurity, the vendor that detected it at 16:00 UTC on 14 March 2025, describes what the commit ran: a memdump against /proc/[pid]/mem of the Runner.Worker process, with the recovered secrets grepped and base64-printed into the workflow log. On public repositories those logs are readable by anyone.
CISA published an alert on 18 March 2025: “A popular third-party GitHub Action, tj-actions/changed-files (tracked as CVE-2025-30066), was compromised. tj-actions/changed-files is designed to detect which files have changed in a pull request or commit. The supply chain compromise allows for information disclosure of secrets including, but not limited to, valid access keys, GitHub Personal Access Tokens (PATs), npm tokens, and private RSA keys. This has been patched in v46.0.1.”
The GitHub Advisory Database records the flaw at CVSS 8.6, affected through 45.0.7, patched in 46.0.1. It puts the blast radius at over 23,000 repositories.
- Untrusted code: The runner executed a third-party Action, and the version tag it was pinned to was moved under it — v1 through v45.0.7 all resolved to the attacker's commit on 14–15 March 2025.
- Broad secret access: Whatever the job held was in the process it ran beside. The dump against /proc/[pid]/mem of Runner.Worker needed no permission to any particular secret, only proximity to all of them.
- Exfiltration risk: This is the bullet that stopped being hypothetical. Access keys, GitHub Personal Access Tokens (PATs), npm tokens and private RSA keys were base64-encoded into workflow logs that are public on public repositories.
- Bad assumption: A version tag was treated as an immutable identity, and a widely used Action as a reviewed one. Neither is a security boundary. Over 23,000 repositories were relying on both.
- Required control: CISA's remedy was to rotate every secret that could have been exposed, not merely to upgrade to v46.0.1. Before the next incident: isolated runners, synthetic fixtures, short-lived scoped credentials, dependencies resolved by immutable commit identity rather than a movable tag, and the expensive jobs held behind the merge.
Visual
A staged CI funnel
Each stage removes a class of failure before spending more time and compute.
The first stage's least glamorous job is secret scanning, and it has the largest measured population behind it. Two organisations counted 2024 independently and neither found the problem shrinking. GitHub reports that “More than 39 million secrets were leaked across GitHub in 2024 alone.” GitGuardian, a scanning vendor counting only public repositories, reports that “This year's findings show no improvement in the fight against secrets sprawl, with 23.8 million secrets leaked on public GitHub repositories in 2024, marking a 25% year-over-year increase.”
The figure that decides CI design is a third one in the same report: 70% of the secrets leaked in 2022 were still active. A credential that reaches a log is not a moment of exposure but a standing one. That is why the funnel's cheapest stage scans for secrets, and why its later stages are built so that a long-lived credential is never in the job to be scanned for.
- 1
Static checks
Formatting, types, linting, dependency policy, and secret scanning.
- 2
Unit and contract tests
Transformations, schemas, invariants, serialization, and API compatibility.
- 3
Small-data integration
Tiny pipelines, fixture datasets, package loading, and deterministic smoke tests.
- 4
Selective model checks
Short training, equivalence, golden cases, or targeted evals based on changed assets.
- 5
Merge evidence
Signed results, artifact identities, approvals, and a declared impact scope.
Steps
Design a change-aware CI policy
Map asset changes to the evidence required before merge.
Step 3 is not the author's preference. NIST's Secure Software Development Framework, SP 800-218, numbers it: “PO.5.1: Separate and protect each environment involved in software development.” Its provenance companion, PS.3.2, asks teams to “Collect, safeguard, maintain, and share provenance data for all components of each software release”. Since 11 March 2024 those practices have carried a signature. CISA's Secure Software Development Attestation Form requires a producer selling to US federal agencies to attest that its environments are secured by “Separating and protecting each environment involved in developing and building software;”. Trust-zone separation is now a thing a named person signs for.
Step 4 has a case attached to it. Codecov's Bash Uploader is a script that customers execute inside their own CI runners. For two months it was not the script Codecov had published. “Our investigation has determined that beginning January 31, 2021, there were periodic, unauthorized alterations of our Bash Uploader script by a third party, which enabled them to potentially export information stored in our users' continuous integration (CI) environments.” An error in Codecov's Docker image creation process had let an attacker extract the credential needed to modify the script, and the altered version exported “any credentials, tokens, or keys that our customers were passing through their CI runner”.
The tampering ran from 31 January 2021 to 1 April 2021. No scanner, no green build and no review caught it. What caught it was a customer who compared the published SHA against the script actually fetched.
The downstream bill was not abstract. On 22 April 2021 HashiCorp disclosed that the GPG private key used to sign the hashes validating its product downloads had been exposed. It had to be rotated and revoked, and the existing releases re-signed. Binding results to artifact identity is step 4 because checksums, and only checksums, ended that incident.
1. Inventory change classes
Code, data logic, schemas, dependencies, policy, evaluation, and documentation.
2. Define impact rules
State which tests and downstream assets each class can affect.
3. Separate trust zones
Restrict secrets, networks, runners, and artifact signing by branch trust.
4. Produce immutable results
Bind test evidence to source revision, environment, inputs, and runner identity.
5. Review the gates themselves
Require explicit approval when acceptance logic or policy changes.
Comparison
Pre-merge and post-merge validation have different trust
The pipeline should not grant the same authority to unreviewed and to protected code. The measured default is that it does. Seven researchers analysed 447,238 GitHub Actions workflows across 213,854 repositories and reported at USENIX Security in 2022: “Our analysis shows that 99.8% of workflows are overprivileged and have read-write access (instead of read-only) to the repository. In addition, 23.7% of workflows are triggerable by a pull_request and use code from the underlying repository.” They also found 99.7% of repositories running some externally developed Action, and 18% running at least one Action with missing security updates.
The vendor eventually moved the floor. On 2 February 2023 GitHub changed the default GITHUB_TOKEN to read-only for new enterprises, organisations and repositories, writing that “Previously, GitHub Actions gets a GITHUB_TOKEN with both read/write permissions by default whenever Actions is enabled on a repository. As a default, this is too permissive, so to improve security we would like to change the default going forward to a read-only token.” New repositories only. The 99.8% was already there.
What the pre-merge column costs when it is wrong was demonstrated on an ML repository. In August 2023 Adnan Khan and John Stawinski IV showed that a fork pull request to pytorch/pytorch could execute attacker code on a self-hosted runner. The mechanism is a default, not a misconfiguration: “By default, when a self-hosted runner is attached to a repository, any of that repository's workflows can use that runner. This setting also applies to workflows from fork pull requests. Remember that anyone can submit a fork pull request to a public GitHub repository.”
Fork pull requests require approval only for accounts that have not previously contributed. A first accepted contribution converts an outsider into someone whose next branch runs unattended. The credentials obtained reached “over 93 repositories within the PyTorch organization”, and Stawinski summarises the reach as follows: “Our exploit path resulted in the ability to upload malicious PyTorch releases to GitHub, upload releases to AWS, potentially add code to the main repository branch, backdoor PyTorch dependencies – the list goes on.” They reported it to Meta's bug bounty programme on 9 August 2023. That is what treating review status as a security boundary buys.
Pre-merge CI
Runs on proposed changes before they enter the protected branch.
- Prioritize fast feedback
- Use synthetic or isolated data
- Restrict secrets and network access
- Block incompatible changes
Post-merge validation
Runs trusted code from the protected branch with broader resources.
- Can access controlled datasets
- May build release candidates
- Produces signed provenance
- Still needs least privilege
Scheduled qualification
Exercises costly or slow evidence on a recurring cadence.
- Large eval suites
- Reproducibility and drift checks
- Dependency and base-image refresh
- Should not replace change-triggered gates
Key idea
A green merge can still hide a changed evaluation rule
If metric code, slice definitions, or acceptance thresholds change in the same pull request as the candidate, the system may pass by moving the goalposts. Evaluation assets need versioning and independent review when they define release criteria.
The CI report should display which gates changed, and whether historical candidates would pass under the new rules. Note where this sits in the rubric. Acceptance logic is neither model code nor infrastructure, so it is exactly the kind of asset that a pipeline testing only what looks like software will never test at all.
Validation logic is production code because it controls what is allowed to ship.
CI protects the protected branch, not the whole lifecycle
Passing CI means the change is suitable to become a candidate under the tested assumptions. It does not authorize production traffic or prove future data quality.
The assumptions are the part that fails quietly. Codecov's customers had green builds for two months while the uploader inside their runners was exporting their credentials. The repositories pinned to tj-actions/changed-files had green builds while the runner dumped its own memory into a public log. In both cases CI passed, and CI was the thing that had been compromised.
Keep the statement narrow: what changed, what was tested, under which trust boundary, and which later gates remain.
Key takeaways
- ML CI must cover feature, label, schema, policy, evaluation, and dependency changes — a `<` to `<=` join edit is exactly the source of dependence that guiding principle 4 of the FDA, Health Canada and MHRA principles requires be considered and addressed.
- Fast deterministic checks should precede selective expensive validation; three of the four sections of the 28-test ML Test Score rubric concern something other than model code.
- Untrusted pull requests should not inherit production secrets or network authority: 99.8% of 447,238 analysed workflows had read-write instead of read-only repository access, and 23.7% were triggerable by a pull_request while running the repository's code.
- Acceptance metrics and gates are behavior-changing assets that need review, because a pull request that moves the threshold and the candidate together passes itself.
- CI evidence should bind results to exact inputs, environment, and runner identity — two months of tampering with Codecov's Bash Uploader ended only when one customer compared the published SHA against the script it had fetched.
- Passing CI creates a candidate; it does not authorize production deployment, and it does not attest to the pipeline itself — over 23,000 repositories were running CVE-2025-30066 with builds that stayed green.