Skip to content
AI.info

Training and optimization

Training and Optimization as Feedback Engineering

Build a system-level map of objectives, data, gradients, updates, validation, and decisions before tuning any optimizer.

By the end you can

A training run is an argument with evidence

Hitting the target exactly is no guarantee of hitting the purpose. A commercial risk-prediction algorithm, one of a class of tools applied to roughly 200 million people a year in the US, was taken apart in Science in 2019. The researchers worked with 6,079 self-identified Black and 43,539 self-identified White patients, observed over 11,929 and 88,080 patient-years. The algorithm predicted what it had been built to predict. What it had been built to predict was cost. “The bias arises because the algorithm predicts health care costs rather than illness, but unequal access to care means that we spend less money caring for Black patients than for White patients,” Obermeyer and his colleagues write in the abstract. Correcting that proxy would raise the share of Black patients receiving extra help from 17.7% to 46.5%. Within days, the New York Department of Financial Services and Department of Health wrote jointly to UnitedHealth Group's CEO about Optum's Impact Pro.

Training is therefore more than parameter updates. It is a controlled process that connects data, an objective, numerical computation, validation evidence, and a decision about what to do next. And the evidence that closes that loop is itself noisy. One 2018 study ran the same reinforcement-learning configuration ten times and changed nothing but the random seed. The authors “perform 10 experiment trials, for the same hyperparameter configuration, only varying the random seed across all 10 trials”. Henderson and his co-authors then split the ten into two sets of five and averaged each set. The two averages come apart. “the variance between runs is enough to create statistically different distributions just from varying random seeds”, and for TRPO on HalfCheetah-v1 the figure caption gives “t = −9.0916, p = 0.0016”. A single run is one sample, not a finding.

Optimization can work exactly as specified while the product moves in the wrong direction.

Example

Four ways a healthy loss curve can mislead

Three of these four have been counted in published work. The numbers are the part a training dashboard cannot show you.

  • Leakage: the validation number can be an artifact of the split, and the frequency of that has been measured. Kapoor and Narayanan surveyed the fields that have taken up machine learning. “Through a survey of literature in fields that have adopted ML methods, we find 17 fields where leakage has been found, collectively affecting 294 papers and, in some cases, leading to wildly overoptimistic conclusions.” That was 2023, and their taxonomy names eight leakage types. A separate group then tested five of those forms across over 400 connectome-prediction pipelines in four datasets. Leakage via feature selection and repeated subjects drastically inflates measured prediction performance, they reported in Nature Communications in 2024, and small datasets make it worse.
  • Class weighting and thresholds: a rare-event detector can improve recall and still bury the operators, because the deployment threshold decides how many alerts arrive. The Joint Commission's Sentinel Event Alert Issue 50, in 2013, estimates that between 85 and 99 percent of hospital alarm signals do not require clinical intervention. In 2014, 2,558,760 unique alarms were recorded from 461 adult intensive-care patients in 31 days — an audible burden of 187 per bed per day. Nurses annotated a sample of them, and Drew and her co-authors report that “88.8% of the 12,671 annotated arrhythmia alarms were false positives”.
  • Augmentation mismatch: heavy blur augmentation lowers training error while erasing small visual defects that matter in production. This is the one case here with no published count attached to it, which makes it the one a team has to instrument for itself.
  • Checkpoint and test-set bias: reporting the best of fifty validation checkpoints reports a maximum over fifty measurements, and a test set that a whole community selects against drifts the same way. In 2019 the CIFAR-10 and ImageNet test sets were rebuilt by following the original collection procedures. “We evaluate a broad range of models and find accuracy drops of 3% - 15% on CIFAR-10 and 11% - 14% on ImageNet.” write Recht and his colleagues. Then the correction was corrected. Engstrom and his co-authors re-measured the measurement in 2020. Correct for statistical bias in the replication procedure, and only about 3.6% ± 1.5% of the original 11.7% ± 1.0% ImageNet drop remains unaccounted for.

Visual

The loop that turns examples into changed behavior

Six stages stand between an example and a changed parameter, and each one can distort the signal on the way through. Sample evidence: select examples, augmentations, masks, and batch composition for the next update. Produce outputs: run the model with a defined mode, precision, and parameter state. Measure an objective: convert outputs and targets into a scalar or weighted set of losses. This is the stage where health-care cost stood in for illness. Compute an update: backpropagate gradients, transform them through an optimizer, and change parameters. Check consequences: evaluate held-out behavior, numerical health, resource use, and failure slices. This is the stage that leakage corrupts silently, in the 17 fields where it has been documented. Decide deliberately: continue, adjust, stop, roll back, or redesign the experiment.

The six stages are not equally instrumented. Stages two and four leave traces in every log. Stages one, three and six are choices made by people, and they are the ones that decide what the traces are worth.

FigureProcess · 6 steps

  1. 1. Sample evidence

    Select examples, augmentations, masks, and batch composition for the next update.

  2. 2. Produce outputs

    Run the model with a defined mode, precision, and parameter state.

  3. 3. Measure an objective

    Convert outputs and targets into a scalar or weighted set of losses.

  4. 4. Compute an update

    Backpropagate gradients, transform them through an optimizer, and change parameters.

  5. 5. Check consequences

    Evaluate held-out behavior, numerical health, resource use, and failure slices.

  6. 6. Decide deliberately

    Continue, adjust, stop, roll back, or redesign the experiment.

Comparison

Three achievements that should never be collapsed

A good run keeps three questions apart. Optimization progress asks whether the chosen training objective improves on sampled updates. Its evidence is loss and gradient behavior, its risk is that the proxy objective is wrong, and its scope is the current parameter trajectory. Generalization asks whether behavior transfers to unseen examples from the intended distribution. Its evidence is untouched validation slices, its risk is leakage or overfitting, and its scope is future examples under assumptions. Product value asks whether the complete system improves an operational outcome under constraints. Its evidence is workflow and outcome measures, its risk is that thresholds or users negate the gains, and its scope is the model plus the surrounding process.

The health-care algorithm is the case where the three come apart in public. On its own objective it did well. The objective was health-care cost, and cost is not illness — the algorithm, its authors write, “predicts health care costs rather than illness”. The failure lived in the third column, not the first. Remedying the proxy would move the share of Black patients receiving additional help from 17.7% to 46.5%. The class of tools involved is applied to roughly 200 million people in the United States each year. No loss curve registered any of this, and the correction did not arrive through one. It arrived by letter, on 25 October 2019, the day the paper appeared in Science. New York State's Department of Financial Services and Department of Health wrote jointly to the CEO of UnitedHealth Group about Optum's Impact Pro.

The middle column fails on its own terms too, and just as invisibly. Collect fresh test sets by the original collection procedures, and a broad range of models loses “11% - 14% on ImageNet”. Not one parameter had been retrained. Optimization progress was untouched. Only the evidence had been replaced, with evidence nobody had selected against.

FigureComparison · 3 columns

Optimization progress

The chosen training objective improves on sampled updates.

  • Evidence: loss and gradient behavior
  • Risk: proxy objective is wrong
  • Scope: current parameter trajectory
  • Example: cross-entropy falls steadily

Generalization

Behavior transfers to unseen examples from the intended distribution.

  • Evidence: untouched validation slices
  • Risk: leakage or overfitting
  • Scope: future examples under assumptions
  • Example: recall holds on later weeks

Product value

The complete system improves an operational outcome under constraints.

  • Evidence: workflow and outcome measures
  • Risk: thresholds or users negate gains
  • Scope: model plus surrounding process
  • Example: fewer missed safety events

Key idea

Write the training contract before launching compute

A training contract records the dataset snapshot, preprocessing code, model version, objective, optimizer state, precision, evaluation protocol, resource budget, and stop conditions. Without it, a run cannot be interpreted reliably. The contract also names what may change during the experiment, because silent changes to data, code, or metrics turn one run into several incompatible experiments.

Parts of that contract are not one author's advice. They have been published as numbered requirements. In 2021 the FDA, Health Canada and the MHRA jointly issued 10 Guiding Principles for Good Machine Learning Practice for Medical Device Development. Principle 3 requires that data sets be representative of the intended patient population. Principle 4 is “Training Data Sets Are Independent of Test Sets”, and it reads: “Training and test datasets are selected and maintained to be appropriately independent of one another. All potential sources of dependence, including patient, data acquisition and site factors are considered and addressed to assure independence.” That is a split policy written by three regulators. It treats independence as something maintained, not something drawn once.

A research venue imposed the same discipline on itself and then measured the effect. The NeurIPS 2019 Call for Papers told authors they would answer every question of a Reproducibility Checklist during submission, and introduced a code-submission policy. Pineau and colleagues reported the outcome: “The number of authors willingly submitting code is quickly increasing, from less than 50% a year ago, to nearly 75%.” In the same program, 173 papers were claimed in the reproducibility challenge, a 92% increase on ICLR 2019, by participants from 73 institutions. And 34% of reviewers said the checklist answers were useful.

The budget belongs in the contract as well. Meta released the full logbook for its OPT-175B run: “the 148 pages of notes the team has taken in the process”. It covers “~90 restarts over the course of training the lineage of this current model”, and records that the finished job took “~33 days of continuous training on 1024 80GB A100s”. The paper counts “at least 35 manual restarts and the cycling of over 100 hosts over the course of 2 months”, plus an estimated “70+ automatic restarts due to hardware failures”. A single answer can cost as much. Hoffmann and twenty-one co-authors trained “over 400 language models ranging from 70 million to over 16 billion parameters on 5 to 500 billion tokens”. The answer was that “for compute-optimal training, the model size and the number of training tokens should be scaled equally”. Chinchilla, which “uses the same compute budget as Gopher but with 70B parameters”, was then trained “on 1.4 trillion tokens”.

This path studies how objectives, gradients, optimizers, data policies, precision, regularization, and compute shape training. It also develops debugging and experiment-design habits. Another path covers architecture design in more depth, and MLOps covers what a team does with a model once it ships. Mathematical proofs appear only when they clarify a practical training decision.

Reproducibility begins with a stable definition of what was actually trained.

Analogy

A greenhouse with several interacting controls

A greenhouse is managed through temperature, humidity, irrigation, and light. Turning one dial changes several measurements, and the desired harvest appears much later.

So a grower who moved three dials on the same morning has no way to say which one the harvest answered. Local signals, delayed outcomes, and interacting interventions are the conditions that require disciplined experiments. They are also the conditions under which a checklist filled in before the run is worth more than an explanation composed after it.

A loss curve is one instrument panel, not the harvest.

Steps

Design the first trustworthy run

Work through this sequence before exploring sophisticated methods. First, state the intended behavior: describe the prediction or representation the model should produce at decision time. Second, choose observable evidence: define targets, masks, splits, and metrics that can support or refute progress. Hold the split to the standard Principle 4 of the FDA, Health Canada and MHRA guiding principles sets, which asks that all potential sources of dependence be considered and addressed rather than assumed away. Third, establish a baseline: measure a simple model or frozen checkpoint under the same protocol. Fourth, set numerical guardrails: record acceptable loss scale, gradient norms, memory use, and invalid-value handling. Fifth, precommit decisions: specify when to stop, what constitutes failure, and which changes justify another run.

Step five is the one an outside body has managed to enforce. The NeurIPS 2019 Call for Papers made authors answer every question of a Reproducibility Checklist at submission time rather than at review time, and paired it with a code-submission policy. Within a year, code submission had moved from less than 50% to nearly 75%. A precommitment is only a precommitment if the form is filled in while the answer is still unknown.

FigureProcess · 5 steps

  1. 1. State the intended behavior

    Describe the prediction or representation the model should produce at decision time.

  2. 2. Choose observable evidence

    Define targets, masks, splits, and metrics that can support or refute progress.

  3. 3. Establish a baseline

    Measure a simple model or frozen checkpoint under the same protocol.

  4. 4. Set numerical guardrails

    Record acceptable loss scale, gradient norms, memory use, and invalid-value handling.

  5. 5. Precommit decisions

    Specify when to stop, what constitutes failure, and which changes justify another run.

Position

One training run is a sample, not a result

A training run has the shape of a procedure. Prepare the data, set an objective, launch the job, read the number that comes out. Written that way, one run reads as an answer. It is one draw from a distribution. Henderson and his co-authors tested exactly that: they “perform 10 experiment trials, for the same hyperparameter configuration, only varying the random seed across all 10 trials”, then “split the trials into two sets of 5 and average these two groupings together”. The two averages separate — “the variance between runs is enough to create statistically different distributions just from varying random seeds” — and the caption for TRPO on HalfCheetah-v1 records “t = −9.0916, p = 0.0016”. No method and no hyper-parameter separated those two groups. Any difference reported between them would have been the seed.

The procedure framing hides a second thing. A run sounds like one continuous process. Meta's OPT-175B logbook fills “the 148 pages of notes the team has taken in the process”, covering “~90 restarts over the course of training the lineage of this current model”. At least 35 of those restarts were manual, with over 100 hosts cycled in two months, and 70 or more automatic restarts are estimated on top of them. The finished job amounted to “~33 days of continuous training on 1024 80GB A100s”. At scale, a run is an object people repair while it moves.

The same error survives past the run and into the test set. Recht and his colleagues rebuilt CIFAR-10 and ImageNet by following the original collection procedures and found “accuracy drops of 3% - 15% on CIFAR-10 and 11% - 14% on ImageNet”. Nothing had been retrained. The models were the same models. What had changed was that the numbers now came from examples nobody had been selecting against. Then the correction was itself corrected. Engstrom and his co-authors showed that once statistical bias in the replication procedure is accounted for, only about 3.6% ± 1.5% of the original 11.7% ± 1.0% ImageNet drop remains unaccounted for. That pair of papers is the discipline this section is arguing for, performed twice: measure, then measure the measurement, then say which part of the gap survives.

None of this says findings are unavailable. It says what a finding actually costs. Hoffmann and twenty-one co-authors trained “over 400 language models ranging from 70 million to over 16 billion parameters on 5 to 500 billion tokens”. That bought one conclusion — “for compute-optimal training, the model size and the number of training tokens should be scaled equally” — and Chinchilla then spent Gopher's compute budget acting on it, at 70B parameters and 1.4 trillion tokens.

So the useful question about a training result is not what it scored. It is how many seeds it survived, which choices were made after the numbers were visible, and whether anyone has tried collecting the test set again.

Key takeaways