MLOps
Model Retirement, Decommissioning, and Evidence Retention
Plan model end-of-life across consumers, routes, jobs, credentials, artifacts, evidence, retention, deletion, and organizational ownership.
By the end you can
- Distinguish deprecation, traffic removal, retirement, deletion, and archival
- Discover direct and hidden consumers before shutting down an ML asset
- Revoke execution authority while retaining proportionate audit and rollback evidence
- Verify that retired models, policies, endpoints, jobs, and credentials can no longer act
The retired model is still sending decisions
The production alias no longer points at the model. The project is closed. But a forgotten weekly batch job still references the immutable version directly, and one partner integration keeps calling an old endpoint.
Retirement is an operational program. It ends authority across every route and every consumer. It also preserves the evidence needed for rollback windows, incidents, audits, and user disputes.
Deprecation windows can be written down, and some are. Kubernetes publishes one with a number in it. Under Rule #4a, deprecated beta API versions “are no longer served 9 months or 3 minor releases after deprecation (whichever is longer)”. That is a promise to consumers, and it buys the hidden batch job time to be found. A retirement plan without a published window discovers its consumers by breaking them. A plan without a verified removal may not discover them for nine years. That is how long an order-routing function Knight Capital Americas LLC had stopped using in 2003 stayed present and callable on its production servers.
Comparison
Retirement choices depend on remaining obligations
End-of-life is not one irreversible delete button. Retaining a tested package for a bounded rollback window, archiving non-executable records for audit, and deleting artifacts are three different obligations with three different endings. Only the third has been written out, with clocks attached, in a federal court order.
Rite Aid is banned from using facial recognition for security or surveillance for five years. That ban comes from the stipulated order in the FTC's case against the company, filed in December 2023 and entered by the court in February 2024. The order then says what deletion actually means. Within 45 days of the effective date Rite Aid must “delete or destroy all photos and videos of consumers used or collected in connection with the operation of a Facial Recognition or Analysis System prior to the effective date of this Order, and any data, models, or algorithms derived in whole or in part therefrom, and provide a written statement to the Commission, sworn under penalty of perjury, confirming that all such information has been deleted or destroyed;”. Within 60 days it must also identify every third party that received that material, instruct each one to delete it, and demand written confirmation of deletion.
Read the three columns against that order and the generic advice acquires edges. Propagating through derivatives is not a nicety: the order reaches the models and algorithms derived from the images, not only the images. Recording completion is a sworn statement to the Commission on a 45-day clock. Exceptions are third-party copies you no longer hold, and the duty there is 60 days, a written instruction, and a written confirmation back. Retention for rollback and archival for evidence end when their stated purpose ends. Deletion ends on a date someone else may have set.
Retain for rollback
Keep a tested executable package during a bounded recovery window.
- Strict access and expiry
- Dependencies remain available
- Rollback evidence preserved
- Ends when newer release is stable
Archive for evidence
Keep non-executable records required for audit, incident, or dispute.
- Immutable and access-controlled
- May retain metrics and decisions
- No production authority
- Retention follows policy
Delete
Remove artifacts or data when no longer authorized or required.
- Propagate through derivatives
- Record completion and exceptions
- May require rebuilt indexes or caches
- Must not erase required evidence prematurely
End-of-life states should be explicit
Deprecation warns consumers and starts migration. Undeployment removes active traffic from one route. Retirement blocks approved execution across the intended system. Deletion removes artifacts or data according to policy. Archival preserves controlled evidence without permitting use. Combining these states encourages accidental reuse or premature destruction.
Two commercial model providers publish this ladder with numbers attached. Microsoft's Foundry Models lifecycle policy, updated in 2026, defines five stages: Preview, Generally Available, Legacy, Deprecated and Retired. A generally available model is given a retirement date programmatically, 18 months from launch. At 12 months it moves to Deprecated — closed to new customers, still served for existing deployments. Microsoft commits to at least 60 days' notice before a GA retirement and 30 days before a preview retirement. It names the official replacement only 90-120 days out, and states that “Retirement dates aren't extendable”. Every inference request to a retired model returns HTTP 410 Gone. That last detail is the important one. The lifecycle state is machine-checkable from outside, by the consumer, without asking the registry what it believes.
OpenAI splits announcement from shutdown in the same way, in four sentences of its API documentation: “We use the term "deprecation" to refer to the process of retiring a model or endpoint. When we announce that a model or endpoint is being deprecated, it immediately becomes deprecated. All deprecated models and endpoints will also have a shut down date. At the time of the shut down, the model or endpoint will no longer be accessible.” Its entries carry both halves. gpt-3.5-turbo-0125, gpt-4-0613 and gpt-4-turbo were deprecated on 22 April 2026, with shutdown on 23 October 2026.
Registries, access systems, schedulers and documentation should agree on the lifecycle state. And the state should be observable from the caller's side, not asserted on the provider's.
Example
Discontinued in 2003, present and callable in 2012
Knight Capital Americas LLC stopped using an order-routing function called Power Peg in 2003. It did not remove it. The SEC's settled order of 16 October 2013 records the state the code was left in: “Upon deployment, the new RLP code in SMARS was intended to replace unused code in the relevant portion of the order router. This unused code previously had been used for functionality called "Power Peg," which Knight had discontinued using many years earlier. Despite the lack of use, the Power Peg functionality remained present and callable at the time of the RLP deployment.”
A 2012 deployment repurposed the flag that had switched Power Peg on. One of eight SMARS servers did not receive the new code. Nothing in the registry-equivalent view was wrong: the function was unused, the project was over, nine years had passed. The 212 parent orders that arrived that morning did not consult any of that.
- Discontinued, not removed: Power Peg stopped being used in 2003 and remained, in the SEC's words, present and callable for nine years.
- Deployment status: a 2012 release repurposed the flag that activated it, and one of eight SMARS servers never received the new code.
- Hidden consumer: 212 incoming parent orders reached that server and triggered the dormant functionality.
- Continued effect: over 4 million executions in 154 stocks, for more than 397 million shares, in approximately 45 minutes, and a $460 million realized loss.
- Repair and price: Knight paid a $12,000,000 civil money penalty; the retirement test that would have caught this is not "is it still used?" but "can it still execute, on every server?"
Visual
The retirement surface
Several independent paths can keep a model active: traffic and schedules; consumers and contracts; authority and access; artifacts and data; evidence and obligations. The surface is only as closed as its least inventoried element. At Knight the surface was eight SMARS servers, and the answer was correct on seven of them. The traffic that found the eighth arrived as 212 ordinary parent orders, not through any route someone still listed as live.
Traffic and schedules
Endpoints, aliases, batch jobs, stream processors, edge packages, and fallbacks.
Consumers and contracts
Applications, analysts, partners, dashboards, and downstream decisions.
Authority and access
Service accounts, secrets, tool scopes, registry permissions, and signing trust.
Artifacts and data
Models, images, indexes, prompts, features, caches, logs, and backups.
Evidence and obligations
Decision records, incidents, approvals, retention, deletion, and appeals.
Steps
Retire an ML system
The plan has five moves: discover use, migrate consumers, remove authority, reconcile late outputs, verify closure. Inventory aliases, direct version references, code, jobs, endpoints, devices and partner contracts. Provide replacement semantics, deadlines, support and a rollback window. Disable routes, schedules, tools, credentials and signing trust. Find late outputs and propagate deletion. Then monitor denied calls, scan references, test that execution is blocked, and assign residual ownership.
Steps four and five are where teams write "deletion recorded" and stop. There is a published definition of what that record contains. NIST rewrote its media-sanitization guidelines in September 2025, and the new revision, SP 800-88r2, withdrew and superseded the December 2014 one. It separates two questions that teams routinely merge. Sanitization verification asks whether the technique completed. Sanitization validation asks whether the target data was effectively sanitized, and ends in an approve or a reject. A technique that ran to completion on the wrong target passes the first and fails the second.
The certificate of media disposition it specifies has fields. The media type and serial number. The sanitization method — clear, purge or destroy — and the technique. The tool used, including its version. The verification method. Then the name, title, date and signature of the person performing verification and validation. That is a closure record with a human being on it.
U.S. law treats the series as the benchmark. Under HHS's breach-notification rule, protected health information counts as unusable where “Electronic media have been cleared, purged, or destroyed consistent with NIST Special Publication 800-88, Guidelines for Media Sanitization, such that the PHI cannot be retrieved.” Model artifacts, feature caches, indexes and backups deserve the same fields: what, by which method, with which tool version, verified how, validated by whom.
1. Inventory consumers
Search aliases, direct versions, code, jobs, endpoints, devices, and partner contracts.
2. Plan migration and fallback
Provide replacement semantics, deadlines, support, and rollback window.
3. Remove execution paths
Disable routes, schedules, tools, credentials, and signing trust.
4. Reconcile and retain evidence
Find late outputs, preserve required records, and propagate deletion.
5. Verify closure
Monitor denied calls, scan references, test that execution is blocked, and assign residual ownership.
Key idea
Retention “just in case” can become permanent ungoverned storage
Keeping every model, dataset, prompt and log indefinitely increases privacy, security, licensing and maintenance risk. Retention should name the purpose, the owner, the access, the expiry, and the deletion or archival action.
Rollback windows should be deliberate and tested. They should not become an excuse to preserve unsupported runtimes forever. A retained artifact with no expiry and no named owner is the same object as Power Peg: unused, unremoved, and still able to act if something reaches it. The sanitization certificate ends with a signature and a date. A retention entry should end the same way.
Evidence retention needs a purpose and expiry just as production execution needs authority.
Case
Six months of logs is a floor, not a schedule
Some retention periods are set by law rather than by taste. Providers of high-risk systems must keep the logs those systems generate, to the extent the logs are under their control, “for a period appropriate to the intended purpose of the high-risk AI system, of at least six months, unless provided otherwise in the applicable Union or national law, in particular in Union law on the protection of personal data”. That is Article 19 of the EU AI Act, covering the logs required by Article 12. It is a floor, not a target.
Other floors are shaped differently, and they are enforced with money. 17 CFR 240.17a-4 requires broker-dealers to preserve the records listed at 240.17a-3(a)(1)-(3), (5), (21) and (22) “for a period of not less than 6 years, the first two years in an easily accessible place”. A further class, at paragraph (b)(4), covers originals of all communications received and copies of all communications sent relating to its business. Those must be kept “not less than three years, the first two years in an easily accessible place”. Note what the rule regulates besides duration: accessibility. Records sitting on cold storage nobody can query are not preserved in the sense the rule means.
The price of missing that is public. On 27 September 2022 the SEC charged 15 broker-dealers and one affiliated investment adviser with failing to preserve off-channel communications from January 2018 through September 2021. The penalties totalled more than $1.1 billion. The same day the CFTC ordered 11 financial institutions to pay over $710 million for the parallel recordkeeping and supervision failures. Gurbir S. Grewal, Director of the SEC's Division of Enforcement, put the reason plainly: “Today's actions – both in terms of the firms involved and the size of the penalties ordered – underscore the importance of recordkeeping requirements: they're sacrosanct. If there are allegations of wrongdoing or misconduct, we must be able to examine a firm's books and records to determine what happened.”
Data-protection law can shorten what is kept. A dispute can lengthen it. That is why the schedule needs a named owner rather than a default.
Retirement is complete when authority is gone
A registry status is only one signal. Closure requires evidence that no approved path can execute the asset: a denied call, a scan that finds no runtime reference, a retired endpoint answering 410 Gone. It also requires that unsupported attempts stay visible.
The final record should state what was deleted, archived, retained, migrated and still owned, plus the date on which remaining evidence expires. That date is frequently not yours to choose. Under Article 18(1) of the EU AI Act, the provider of a high-risk AI system must keep its documentation at the disposal of national competent authorities “for a period ending 10 years after the high-risk AI system has been placed on the market or put into service”. The list of documents is long: the Article 11 technical documentation, the Article 17 quality-management-system documentation, documentation of changes approved by notified bodies, notified bodies' decisions and documents, and the Article 47 EU declaration of conformity. The clock starts at placing on the market, not at retirement. A system withdrawn early carries the remainder of those ten years with it.
Residual ownership is a legal question as much as an organizational one, and the law asks it in the hardest form: what happens when the owner stops existing. Article 18(2) provides that “Each Member State shall determine conditions under which the documentation referred to in paragraph 1 remains at the disposal of the national competent authorities for the period indicated in that paragraph for the cases when a provider or its authorised representative established on its territory goes bankrupt or ceases its activity prior to the end of that period.” A retirement plan that cannot name who holds the evidence after the team, the product, or the company is gone has closed the traffic and left the obligation open.
Key takeaways
- Deprecation, undeployment, retirement, archival, and deletion are different lifecycle states: Microsoft's Foundry policy names five (Preview, Generally Available, Legacy, Deprecated, Retired) and proves the last with HTTP 410 Gone.
- Hidden consumers reference immutable versions, jobs, endpoints, devices, or partner contracts — Power Peg waited on one of eight SMARS servers until 212 parent orders reached it.
- Retirement removes authority, not merely visible traffic: discontinued in 2003, Power Peg still produced over 4 million executions in approximately 45 minutes in 2012 and a $460 million realized loss.
- Deletion propagates: the Rite Aid stipulated order reaches “any data, models, or algorithms derived in whole or in part therefrom” within 45 days, and third-party copies within 60, with sworn and written confirmation.
- Retention floors are law, not taste: at least six months of Article 12 logs under Article 19 of the AI Act, six years and three years under 17 CFR 240.17a-4, and more than $1.1 billion in SEC penalties on 27 September 2022 for failing the latter.
- Closure is a record: a certificate naming method, tool version, verification, validation and a signatory, and an Article 18 clock running ten years from placing on the market.