MLOps as a Control System

A model can be statistically strong and still fail as a service. On 2 November 2021 Zillow Group announced it would wind down the business its pricing model existed to run. Zillow Offers bought houses at algorithmically set prices. That quarter carried a write-down of inventory of approximately $304 million in the Homes segment, because homes had been bought above what the company itself later estimated they would resell for. The wind-down would cut roughly 25% of Zillow Group's workforce.

Rich Barton, co-founder and CEO of Zillow Group, gave the reason in the release itself: “We've determined the unpredictability in forecasting home prices far exceeds what we anticipated and continuing to scale Zillow Offers would result in too much earnings and balance-sheet volatility.” The annual filing to the SEC records the same decision in the flat language of an audited document: “On November 2, 2021, we made the determination to wind down Zillow Offers operations within the Homes segment.”

Nothing in that outcome was an outage. Prices were produced, delivered, and acted on. What failed was everything wrapped around the prediction. How much capital a price commitment put at risk. How fast the business scaled on the strength of it. How quickly anyone could tell that forecast error had stopped being a metric and become a balance-sheet problem.

MLOps begins where a notebook ends. It coordinates the evidence, software, infrastructure, people, and policies that make a prediction usable in a real process. The aim is not maximal automation. It is controlled change with visible consequences — and, as $304 million of written-down inventory shows, a stop condition that someone is authorised to pull.

The visible endpoint rests on several layers, and they fail independently. The decision contract fixes who uses the output, what action follows, and which errors are unacceptable. The learning assets are the datasets, labels, features, code, parameters, prompts, and evaluation suites. The delivery machinery is the pipelines, registries, release gates, runtime contracts, and rollback paths. The operational evidence is the logs, metrics, traces, quality measurements, incidents, and audit records. The human organization is the owners, reviewers, on-call responders, approvers, and downstream consumers.

The cases in this lesson are useful because each one broke in a different layer while the others held. Zillow Offers failed in the decision contract: the price was computed and honoured, and the exposure it created was the problem. Knight Capital Americas failed in the delivery machinery on 1 August 2012, when new code reached seven of eight servers. The commercial risk-prediction algorithm dissected by Obermeyer and colleagues failed in the learning assets, because the label it was trained on was health care costs rather than illness. The Epic Sepsis Model at Michigan Medicine failed in the operational evidence, where an AUC of 0.63 was measured only years after deployment, by an outside team. Ofqual's 2020 standardisation model failed in the human organization, where the reversal came four days after results were issued.

No single dashboard watches five layers at once. That is why the map has to be drawn before the tools are chosen.

On 1 August 2012 Knight Capital Americas deployed new RLP code to its SMARS order-routing system. Every server stayed up. The firm lost over $460 million in 45 minutes. How it happened is not a matter of opinion: the SEC set it out in an order against the firm on 16 October 2013. Five separate domains failed, and each is recognisable in an ML system.

A single dashboard rarely proves that an ML product is healthy. The Epic Sepsis Model shows the three notions coming apart inside one deployed system.

Service health held throughout. The model was live across US hospitals, scoring patients and firing alerts inside the electronic health record. Nothing about its availability was in question.

Model quality was a separate matter, and it took an outside team to measure it. Wong and colleagues studied 27,697 patients and 38,455 hospitalizations at Michigan Medicine, with sepsis in 2,552 of them — 6.6%, reported as 7% in the abstract. They found an AUC of 0.63 (95% CI, 0.62-0.64). Their conclusion, published in JAMA Internal Medicine in 2021, does not hedge: “This external validation cohort study suggests that the ESM has poor discrimination and calibration in predicting the onset of sepsis.”

Product outcome is the third measurement, and it is the one about workflow capacity. The alerting threshold was a score of 6, chosen by the hospital's own operations committee and inside the 5-8 range the model's developer suggested. At that threshold the model alerted on 18% of all hospitalizations, 6,971 of 38,455. Sensitivity was 33%. It did not identify 1,709 patients with sepsis, 67% of them. Clinicians would still have to evaluate 8 patients to identify one who developed sepsis. A second, independent group reached the same shape of conclusion elsewhere: across 145,885 encounters in 2023 at two county emergency departments, reported in JAMIA Open in 2024, sensitivity was 14.7% and positive predictive value 7.6% within a 6-hour window.

A service-health dashboard would have been green for the whole of that. Discrimination is one measurement. Alert burden on the people who must respond is another. Neither is inferable from uptime.

The serving loop converts current inputs into outputs within seconds or milliseconds. The monitoring loop aggregates telemetry over minutes or hours, while the learning loop may retrain weekly. Governance and product review often move more slowly still.

The gap between those clocks is measurable in the cases above. Knight Capital Americas' serving loop did its damage in 45 minutes on 1 August 2012; the governance loop closed with the SEC's order on 16 October 2013. The Epic Sepsis Model served predictions in real time, and the AUC of 0.63 that described their quality was published in 2021, by researchers outside the vendor. Zillow Group's pricing loop ran continuously; the correction was a single decision recorded on 2 November 2021, against approximately $304 million of inventory already bought.

A reliable design names these clocks explicitly. Otherwise a fast service can go on making a degraded decision for days, or for years, because the only useful label arrives after the operational damage.

Use this exercise before selecting tools or platforms, and test each step against a case that has already failed there.

Start with the decision: write the action that changes because the model exists. For Zillow Offers that action was buying a house at a stated price. That is why the failure arrived as approximately $304 million of written-down inventory rather than as a metric regression.

Trace every input: record sources, clocks, transformations, and availability conditions. The Epic Sepsis Model's inputs were fine. What nobody had traced was the evidence path, which is why the AUC of 0.63 was measured at Michigan Medicine by an outside team in 2021 rather than by the deployment itself.

Mark every mutable asset: model versions, configuration, policies, thresholds, and routing rules. Knight Capital Americas had eight SMARS order-routing servers and a repurposed flag. The SEC found no written code-deployment procedures and no requirement for a second technician to review deployment.

Add evidence paths: show where service, data, model, and outcome signals are produced. The sepsis alert threshold of 6 was set by a hospital operations committee, inside a developer-suggested 5-8 range. The number that determined alerts on 18% of hospitalizations lived in a committee minute, not in the model.

Assign owners and recovery actions: name who can stop, roll back, repair, or retire each component. Ofqual's chair could reverse a national grading model on 17 August 2020, four days after results. Ask who holds that authority for your system, and whether they could exercise it within four days.

Every later lesson asks the same three questions: What can change? What evidence makes that change acceptable? How does the team recover when the assumption fails?

These are not an author's framework. In the United States they are law for one class of ML system. Congress added section 515C to the Federal Food, Drug, and Cosmetic Act in 2022. Under it, a supplemental application “shall not be required for a change to a device ... if such change is consistent with a predetermined change control plan” the agency has approved. A manufacturer may therefore change a model without a new submission — but only by having declared in advance what could change and how it would be checked.

The FDA issued the final guidance on those plans for AI-enabled device software on 4 December 2024. Its summary sets out the same three questions in order: “This guidance recommends that a PCCP describe the planned AI-DSF modifications, the associated methodology to develop, validate, and implement those modifications, and an assessment of the impact of those modifications.”

Tools matter only after those questions have concrete answers. A registry, orchestrator, or dashboard cannot supply the missing product contract.

Sculley and his co-authors named the effect that makes this hard. They call it the CACE principle: “Changing Anything Changes Everything”. It applies to input signals, and equally to hyper-parameters, learning settings, sampling methods, convergence thresholds and data selection, because “No inputs are ever really independent.” That is why the three questions above — and the plan the FDA requires — are asked of the whole system rather than of the model.