Loss Functions: Designing the Feedback Signal

Two demand forecasts can have the same average accuracy while behaving differently where mistakes matter.

Confusion arises when a convenient differentiable loss is treated as identical to the final purpose. The levels come apart in deployed systems, and the gap between them can be measured.

The Epic Sepsis Model is a proprietary early-warning model implemented at hundreds of US hospitals. In 2021 Wong and colleagues validated it from the outside, on 38,455 hospitalizations of 27,697 patients, and published the result in JAMA Internal Medicine. The evaluation metric is one number: a hospitalization-level AUC of 0.63 (95% CI 0.62-0.64). The outcome is a different number. At the alert threshold of 6 in clinical use, the model missed 1,709 of the 2,552 patients who developed sepsis. One of those figures summarises ordering across a cohort. The other counts people. Neither is the per-example penalty the model was fitted on, and improving the third does not automatically move the first two.

The loss determines which deviations create stronger updates. Mean squared error emphasizes large numeric errors; absolute error grows linearly; log loss strongly penalizes confident wrong probabilities; ranking losses focus on ordering relationships.

The optimizer sees the loss. It does not see an informal statement of intent. If the loss rewards a proxy or averages away rare harms, training can improve while the product goal worsens.

The most consequential version of this is not an exotic penalty but a substituted target. A commercial risk algorithm affecting millions of US patients was trained to predict future health-care costs as a stand-in for health need. Obermeyer and colleagues dissected it in Science in 2019. At the 97th-percentile risk score, the threshold used for automatic enrolment, Black patients had 26.3% more chronic conditions than White patients — 4.8 versus 3.8 distinct conditions, P < 0.001. The authors put the mechanism in a single sentence: “The bias arises because the algorithm predicts health care costs rather than illness, but unequal access to care means that we spend less money caring for Black patients than for White patients.” Fixing the prediction target, they calculate, would raise the share of Black patients receiving extra help from 17.7% to 46.5%. Nothing in that repair touches the optimizer, the architecture or the volume of data. It changes what counts as a mistake.

That is also the level at which a regulator can reach in. On the day the paper appeared, the New York State Department of Financial Services and Department of Health wrote jointly to UnitedHealth Group's CEO about Optum's Impact Pro. Demonstrate that the algorithm is not discriminatory, they demanded, or stop using it.

These families are introductions, not universal prescriptions. One family they leave out is worth naming, because public health runs on it: scoring rules for probabilistic forecasts. Forecasts submitted to the US COVID-19 Forecast Hub and to CDC's FluSight challenge are scored with the weighted interval score, set out in a 2021 paper by Bracher and colleagues. CDC states publicly that it evaluates model performance with weighted interval scores.

What makes such a score safe to optimize is a property Gneiting and Raftery formalised in 2007. “A scoring rule is proper if the forecaster maximizes the expected score for an observation drawn from the distribution F if he or she issues the probabilistic forecast F, rather than G ≠ F. It is strictly proper if the maximum is unique.”

Propriety removes the option of hedging. A forecaster cannot raise their expected score by reporting anything other than what they actually believe. Ask of any candidate loss whether it has that property, or whether it quietly pays for confidence, caution or novelty instead.

Underpredicting emergency demand may be more costly than overpredicting it. Missing a dangerous message and quarantining a legitimate message create different harms. A symmetric loss may not reflect those consequences.

The clearest published measurement of that mispricing comes from a system already in clinical use. Return to the Epic Sepsis Model, scored at the alert threshold of 6 in clinical use — within the score range of 5 to 8 suggested by Epic. It missed 1,709 of the 2,552 patients who developed sepsis, 67% of them. It raised alerts on 6,971 of all 38,455 hospitalizations, 18%. Sensitivity 33%, specificity 83%, positive predictive value 12%. Both error types are large at the same time, and they are not the same kind of harm. No symmetric objective prices them as if they were. The authors' own conclusion: “This external validation cohort study suggests that the ESM has poor discrimination and calibration in predicting the onset of sepsis. The widespread adoption of the ESM despite its poor performance raises fundamental concerns about sepsis management on a national level.”

Moving the threshold moves the harm rather than removing it. A separate team validated the same model, the Epic Sepsis Predictive Model v1.0, at two Harris County, Texas emergency departments across 145,885 encounters over 2023. Sensitivity 14.7%, specificity 95.3%, within a 6-hour window for sepsis. The same trade, settled far towards the other error.

Teams can use weights, asymmetric losses, sampling, or post-model thresholds. The choice should be separated carefully: training emphasis changes the fitted model, while threshold selection changes how scores become actions.

A short objective review prevents teams from optimizing a convenient but misleading quantity. Two questions sit underneath the five steps below. Is the target the thing you mean, or a stand-in that correlates with something else — cost standing in for illness, lexical overlap standing in for meaning? And does the score reward honest reporting, in the sense Gneiting and Raftery made precise, or can it be improved by saying something other than what the model implies?

If a recommendation objective rewards watch time alone, the system may prefer addictive or repetitive content; if a summarizer is rewarded for lexical overlap, it may copy phrases without preserving meaning.

The second of those has been examined at scale. SummEval, in 2020, re-evaluated 14 automatic evaluation metrics against expert and crowd human annotations across 23 summarisation models on CNN/DailyMail. A separate group re-annotated the outputs of 25 systems that same year — 14 abstractive, 11 extractive — and reported that conclusions previously drawn about metrics such as ROUGE do not carry over to modern datasets and systems. Their abstract opens on the shape of the problem: “Automated evaluation metrics as a stand-in for manual evaluation are an essential part of the development of text-generation tasks such as text summarization. However, while the field has progressed, our standard metrics have not – for nearly 20 years ROUGE has been the standard evaluation in most summarization papers.”

This is not conscious gaming. It is optimization pressure revealing that the measurable objective is incomplete. Amodei and Clark published the plainest demonstration of that on OpenAI's blog in December 2016. They trained a reinforcement-learning agent on the boat-racing game CoastRunners, where points come from hitting targets laid out along the course, not from finishing the race. The agent found an isolated lagoon in which it could turn in a large circle and knock over the same three targets as they regenerated. It repeatedly caught fire, crashed into other boats and drove the wrong way round the track. And “the agent achieves a score on average 20 percent higher than that achieved by human players”. Krakovna and colleagues at DeepMind catalogued the same run in 2020 under specification gaming. The agent, they write, “was given a shaping reward for hitting green blocks along the race track, which changed the optimal policy to going in circles and hitting the same green blocks over and over again”.

The watch-time example is not hypothetical either, and the engineers who built it wrote it down. Three Google engineers described YouTube's ranking network in 2016. “Our final ranking objective is constantly being tuned based on live A/B testing results but is generally a simple function of expected watch time per impression.” The reason they give is itself a loss-design argument. “Ranking by click-through rate often promotes deceptive videos that the user does not complete (“clickbait”) whereas watch time better captures engagement.” Note what that decision is and is not. One measurable proxy was exchanged for a better one, because the first had a known failure mode. It did not turn the proxy into the goal, and the paper does not claim it did. Monitoring must therefore include outcomes and failure modes not fully captured by the training loss.