Choosing the Learning Strategy Under Real Constraints

These common situations point toward different first candidates. Hybrid designs remain possible.

The first branch — reliable targets exist — is the one that fails most quietly. A column that is present in the data is not the same thing as the target you want. A commercial health risk algorithm used across US hospitals was trained on future health-care costs, a convenient proxy for health need. Less is spent on Black patients with equal illness. So the model learned that those patients were healthier. Obermeyer and colleagues dissected it in Science in 2019. The Algorithmic Bias Playbook, hosted by the US Federal Trade Commission, has a name for the failure — “label choice bias” — and is precise about what did not go wrong: “it predicted a biased target very well”.

The size of the error was measurable, which is the point. The abstract gives it in one line: “Remedying this disparity would increase the percentage of Black patients receiving additional help from 17.7 to 46.5%.” Regulators moved on the same algorithm the same week. On 25 October 2019 the New York Department of Financial Services and Department of Health wrote jointly to UnitedHealth Group about Optum's Impact Pro. Nothing in the pipeline was broken. The target was chosen because it was already in the data. That single decision is what “audit target provenance” and “do not invent labels by convenience” are asking you to make explicit before anything is fitted.

Every approach exchanges one scarce resource for another. Direct labels cost annotation; weak labels cost source design; self-supervision costs objective and compute; reinforcement learning costs interaction and safety; transfer costs trust in the source.

Write these assumptions as risks that can be tested. A strategy is stronger when its weakest assumption has a clear experiment or fallback.

Google Flu Trends is what an untested weakest assumption looks like at scale. The model was fitted by matching 50 million search terms to 1,152 CDC data points. One assumption carried the whole design: that search behaviour is a stable proxy for illness. It was never given an experiment or a fallback. From August 2011 the model missed high in 100 of 108 weeks. In February 2013 it predicted more than double the CDC's proportion of doctor visits for influenza-like illness. Lazer and colleagues set out the diagnosis in Science in 2014: “In short, the initial version of GFT was part flu detector, part winter detector.” The trivial fallback was never beaten either. Three-week-old CDC data projected current flu prevalence better than the model did. An independent epidemiological reassessment in PLOS Computational Biology in 2013 found that the model had completely missed the first wave of the 2009 A/H1N1 pandemic and had overestimated the 2012/2013 A/H3N2 season.

Some risk survives every test a team actually runs. D'Amour and colleagues at Google gave that risk a name: underspecification. They trained 50 ResNet-50 models on ImageNet through identical pipelines, differing only in the random seed, and reported it in the Journal of Machine Learning Research in 2022. On the ImageNet validation set the fifty were interchangeable: 75.9% top-1 accuracy, standard deviation 0.11. On stress tests they were not. Variation across the ensemble on some ImageNet-C corruption tasks was an order of magnitude larger than variation on the clean test set, and the two were not usefully correlated. So the differences could not be explained away as some models simply being better trained. A pipeline that pins down everything the team wrote down still leaves the model free to choose which shortcut to lean on. The held-out test set cannot see the choice.

Use this as a conversation starter rather than an automatic classifier of projects.

Step 5 is the step teams skip, because a vendor model arrives with the claim already made. Epic's proprietary sepsis prediction model arrived that way. Wong and colleagues at Michigan Medicine tested it anyway, on 27,697 patients and 38,455 hospitalisations, and published in JAMA Internal Medicine on 21 June 2021. The abstract gives the number: “The ESM had a hospitalization-level area under the receiver operating characteristic curve of 0.63 (95% CI, 0.62-0.64).” It alerted on 18% of hospitalisations and missed 67% of sepsis cases. A second, unrelated validation in JAMIA Open covered 145,885 encounters at two county emergency departments in 2023. It found sensitivity of 14.7% and a median alerting lead time of 0 minutes. On the median alert, the model told the clinicians something they already had. Two teams, two datasets, one conclusion that a target-only baseline would have produced before deployment rather than after.

Step 6 has a literature of its own. Ferrari Dacrema and colleagues examined 18 neural recommendation methods from top conferences in 2019: only 7 could be reproduced with reasonable effort, and 6 of those 7 were beaten by simple nearest-neighbour or graph heuristics. It won best paper at RecSys that year. A year later Rendle and colleagues at Google re-ran the experiments that had launched neural collaborative filtering. Their abstract opens: “First, we show that with a proper hyperparameter selection, a simple dot product substantially outperforms the proposed learned similarities.” Two independent groups, in consecutive years, counting the same thing.

A factory need not choose one learning family forever. Each stage below exists because the previous one exposed a limitation it could not fix, which is the only order in which added machinery can be evaluated at all.

A concise memo makes assumptions and alternatives reviewable before a large build begins.

Step 4 is where memos go thin, because “we will compare against a baseline” is a sentence that costs nothing. Dressel and Farid paid for it. In 2018 they set COMPAS — a commercial recidivism tool using 137 features — against the cheapest comparisons available, and published the result in Science Advances. Their abstract: “We further show that a simple linear predictor provided with only two features is nearly equivalent to COMPAS with its 137 features.” The two features were age and number of prior convictions. They also ran the human baseline that most memos never mention. Untrained people recruited online reached a mean accuracy of 62.1% and a median of 64.0%, no less accurate than the deployed product. The rules baseline held as well. Independently, Angelino and colleagues produced certifiably optimal sparse rule lists about as accurate as COMPAS on the Broward County data.

Four comparisons, one dataset, and a number for each: that is what step 4 is asking for. A memo that names its baselines with figures can be argued with before the build; a memo that names them as a category cannot.

A target may be unobservable, a policy may create the labels, a simpler rule may already meet the need, or errors may be too costly for available evidence. Declining deployment can be the most rigorous conclusion.

England ran that argument at national scale in the summer of 2020. Ofqual's own research report on that year's A levels analysed 457,420 entries and records what the standardisation model did to them. It produced a grade below the teacher's centre assessment grade for 39% of entries, and above it for just over 2%. 10.3% of candidates were three or more grades down in total. On 17 August 2020 Ofqual instructed awarding bodies to reissue the higher of the two grades, which withdrew the model in the middle of its deployment. The Office for Statistics Regulation reviewed the episode independently and, on 2 March 2021, put the conclusion to the people who commission such systems in one sentence: “A statistical model might not always be the best approach to meet your need.” The failure was not in the fitting. It was in never having asked whether a model was the right instrument for the decision.

The capstone treats a no-model recommendation as successful when it follows a clear analysis of feedback, assumptions, alternatives, and risk.