From Product Goals to Trainable Objectives

A widely used commercial health-risk algorithm was taken apart, level by level, in Science in 2019. Obermeyer and colleagues did the dissecting. Nothing below is hypothetical. The whole failure is one substitution between two levels.

A product goal is usually delayed, partly observed, and influenced by people. An optimizer needs a numerical signal available for each batch or sequence.

Whoever maps one level onto the next is making a modeling decision. It should stay visible, because each level can fail while another still looks successful. The algorithm New York's regulators would later name as Optum's Impact Pro broke between the top of the stack and the bottom. Illness was the outcome. Health care cost was the training target. Nothing in between recorded that a substitution had been made, so every level below the swap kept reporting success.

Three components do different jobs. A direct outcome sits close to the real consequence and arrives late, sparse, and contaminated by policy changes. A surrogate objective is trainable and produces gradients every batch, but it tracks the desired behavior only under assumptions someone has to state. A guardrail is a condition that must not be traded away for average performance. It is frequently non-differentiable, which is exactly why it goes missing at model selection.

The Cardiac Arrhythmia Suppression Trial is what it costs to let the second stand in for the first. The surrogate endpoint was ventricular ectopy after myocardial infarction. Drugs were selected because they suppressed it.

The trial randomised 1,498 patients to encainide or flecainide or their placebos. Of those, 857 were assigned to encainide or its placebo: 432 to active drug and 425 to placebo. The other 641 were assigned to flecainide or its placebo: 323 to active drug and 318 to placebo. After a mean follow-up of 10 months, 89 patients had died. Of those deaths, 59 were arrhythmic: 43 receiving drug against 16 receiving placebo, P = 0.0004. Echt and colleagues reported it for the CAST Investigators in the New England Journal of Medicine in 1991. Their conclusion reads: “There was an excess of deaths due to arrhythmia and deaths due to shock after acute recurrent myocardial infarction in patients treated with encainide or flecainide.”

The American College of Cardiology keeps its own record of the trial, which the NHLBI sponsored and which enrolled 2,309 patients. It states the harm as a ratio — “Encainide and flecainide caused increased cardiovascular mortality with a relative risk of 2.5” — and concludes that suppressing asymptomatic ventricular arrhythmias after myocardial infarction “does not improve survival and can increase mortality”.

The surrogate moved in the intended direction the whole time. The direct outcome moved the other way, and the encainide and flecainide arms were stopped early for excess mortality. Imperfect alignment is not a caveat you note and move past. It is the entire result.

A changed objective creates a new experiment, even when the architecture stays fixed. Record the reason, the expected effect, and the metrics that should move.

Stopping is part of the same discipline. In the Cardiac Arrhythmia Suppression Trial the encainide and flecainide arms were halted early for excess mortality. That decision was taken against the objective, not against the model that optimized it. The revision that matters is rarely a new hyperparameter. It is the moment someone writes down that the quantity being improved was the wrong quantity, and says on what evidence.

A contractor rewarded only for finishing quickly will finish quickly. Speed may increase while safety checks, material quality, and whatever it costs to keep the building standing later are neglected.

The contractor at least decides to cut those corners. An optimizer decides nothing, so whatever the objective failed to mention is not being neglected. It was never in the room. One measurable proxy cannot encode every valued outcome.

YouTube's recommender ran on clicks, and then it did not. Covington and colleagues wrote the switch down in 2016, in the section of the paper on ranking. The objective first: “Our final ranking objective is constantly being tuned based on live A/B testing results but is generally a simple function of expected watch time per impression.” Then the reason the earlier proxy was abandoned: “Ranking by click-through rate often promotes deceptive videos that the user does not complete (“clickbait”) whereas watch time better captures engagement”.

Nothing there says the click model was inaccurate. It was accurate, and it bought behavior nobody wanted. Notice also what replaced it. Expected watch time per impression is another proxy, carrying its own assumptions about what a completed video means. Proxies get replaced by proxies, not by the goal.

This exercise exposes missing links before they become expensive experiments. Name the consequence in ordinary language. Specify the decision: how outputs turn into actions, rankings, abstention or review. List the measurable evidence, including the slices that could falsify the proposed benefit. Select a trainable target from the labels, preferences, sequences or rewards actually available at training time. Add guardrails: cost, fairness checks, latency, capacity, abstention. Then predict the gaming routes, the shortcuts that would reduce loss without improving the intended outcome.

Step six is the one that gets skipped, so here is a gaming route with a number attached. Networks trained to find pneumonia learned to find the hospital. Zech and colleagues trained convolutional neural networks across 158,323 chest radiographs from three hospital systems. What they reported in PLOS Medicine: “CNNs were able to directly detect hospital system of a radiograph for 99.95% NIH (22,050/22,062) and 99.98% MSH (8,386/8,388) radiographs”. That capability was worth something to the loss, because disease was not evenly spread: “The prevalence of pneumonia was high enough at MSH (34.2%) relative to NIH and IU (1.2% and 1.0%) that merely sorting by hospital system achieved an AUC of 0.861 (95% CI 0.855-0.866) on the joint MSH-NIH dataset”. Guessing the hospital, and nothing about the lungs, scored 0.861. The best internal model reached an AUC of 0.931. Tested externally, it fell to 0.815. The authors' conclusion: “CNNs robustly identified hospital system and department within a hospital, which can have large differences in disease burden and may confound predictions.”

The mechanism reproduces in other tasks. DeGrave and colleagues, writing in Nature Machine Intelligence, report that “we demonstrate that recent deep learning systems to detect COVID-19 from chest radiographs rely on confounding factors rather than medical pathology, creating an alarming situation in which the systems appear accurate, but fail when tested in new hospitals”. The US Agency for Healthcare Research and Quality cites that work. It names the remedy as “better training data, improved choice in the prediction task, and external validation of the AI system prior to dissemination and implementations in different hospitals”. Note the middle item. The choice of prediction task is listed alongside the data, not underneath it.

Five layers, and each one should explain how evidence flows to the next. The operational outcome is what should improve for users or the physical system. The decision policy turns predictions into action, abstention, review or ranking. The evaluation measures test the intended behavior offline and online. The model target is what the system is asked to estimate or rank. The training objective is the differentiable quantity that actually produces an update.

The Epic Sepsis Model lets you read three of those layers off a published paper. The decision policy is an alert at an ESM score of 6 or higher. The evaluation measure, when Wong and colleagues tested it at Michigan Medicine, was a hospitalization-level area under the receiver operating characteristic curve of 0.63 (95% CI, 0.62-0.64). The model target is sepsis. The training objective is the layer a hospital using it cannot inspect at all, because the model is proprietary. That is the point of drawing the stack. A layer nobody can see is still a layer that decides what the system does.

If every disappointing result leads to another loss term, the objective can become an opaque patchwork. Prefer a small, interpretable stack and remove terms that no longer earn their complexity.

Earning it is a measurement, not an argument. Coste and colleagues rebuilt that same setup for ICLR 2024, added 25% label noise, and reported that conservative ensemble objectives improved performance by up to 70% for best-of-n sampling. Their abstract restates what they were building on: “Gao et al. (2023) studied this phenomenon in a synthetic human feedback setup with a significantly larger "gold" reward model acting as the true reward (instead of humans) and showed that overoptimization remains a persistent problem regardless of the size of the proxy reward model and training data used.” The extra structure was kept because it was scored against a standard held outside the proxy. A term added because the last experiment disappointed has no such record. It also has no criterion for ever being removed.