Continuous Training and Retraining Policy

A demand model retrains every night to remain current. A source outage leaves one category underrepresented. The pipeline still qualifies and deploys the candidate because the job completed and the aggregate metric remains acceptable.

Continuous training should automate how evidence is gathered and candidates built, not assume that newer is better. That is not only an engineering opinion. On 27 October 2021 the FDA, Health Canada and the UK Medicines and Healthcare products Regulatory Agency jointly issued guiding principles for machine learning in medical devices. Principle 10 is titled “Deployed models are monitored for performance and retraining risks are managed”. It does not describe training after deployment as an upgrade. It describes it as an exposure to be controlled: “Additionally, when models are periodically or continually trained after deployment, there are appropriate controls in place to manage risks of overfitting, unintended bias, or degradation of the model (for example, dataset drift) that may impact the safety and performance of the model as it is used by the Human-AI team.”

Read the nightly pipeline against that sentence. The periodic training is there. So is the degradation it names: an unrepresentative category, arriving through the data source rather than the code. What is missing is the control. A job that completed and an aggregate metric that stayed acceptable are the two things a degraded candidate produces without effort.

A safe loop separates observation, candidate creation, qualification, and release. Observe data quality, drift, labels, outcomes, incidents and business change. Decide whether to train, under a trigger policy with cooldown, minimum evidence and owner review. Build the candidate with data, code, configuration, environment and evaluation plan frozen. Qualify against baseline, slices, cost, stability and the operational contract. Then promote progressively, retain the current model, or change a different component.

The separation exists so that each stage can refuse something the previous stage produced. A stage that cannot reject its input is not a stage. It is a conveyor with a name.

Scheduled retraining is simple and can capture predictable seasonality. Event triggers respond to new labels, source changes, or policy events. Evidence triggers react to measured deterioration, drift, or opportunity.

These triggers create candidates, not automatic truth. Qualification must check data validity, comparison fairness, operational readiness, and whether a smaller intervention — recalibration or a threshold update — would address the issue.

The line between a change you declared in advance and one you improvised is no longer only an engineering preference. Article 43(4) of Regulation (EU) 2024/1689, the EU AI Act, provides: “For high-risk AI systems that continue to learn after being placed on the market or put into service, changes to the high-risk AI system and its performance that have been pre-determined by the provider at the moment of the initial conformity assessment and are part of the information contained in the technical documentation referred to in point 2(f) of Annex IV, shall not constitute a substantial modification.”

Run that backwards and it is a specification for a trigger policy. Retraining whose scope the provider wrote into the technical documentation under point 2(f) of Annex IV, before the initial conformity assessment, sits inside the assessment already made. Retraining outside that written scope is a substantial modification. The scope has to exist as text, before deployment. An undocumented nightly job has no such text. Its trigger was never declared, so every candidate it ships is justified after the fact by its own result.

A pretrial release decision leaves a hole in the data. You learn what happened to the defendants a judge released. You learn nothing about the ones the judge detained. Kleinberg and four co-authors met that hole in a file of 1,460,462 New York City arrest cases covering 1 November 2008 to 1 November 2013, of which 758,027 — the cases subject to a pretrial release decision — formed the analysis sample. They stated the obstacle in one sentence: “We only observe crime outcomes for released defendants, not for those judges detained.” A loop that retrains on outcomes inherits whatever the incumbent policy allowed to happen.

The policy should define when to create a candidate and when not to deploy it. Define the trigger evidence: schedules, label volume, drift, performance, incidents, domain events. Set guardrails: data quality, cooldown, minimum coverage, feedback-loop review. Choose the intervention: retrain, recalibrate, change policy, repair data, or do nothing. Qualify fairly against controlled baselines, with fresh evaluation, slices, stability and cost. Release with churn control: progressive exposure, rollback, monitoring and minimum-improvement rules.

Those five steps have a published counterpart. In October 2023 the FDA, Health Canada and the MHRA jointly set out five principles for predetermined change control plans — Focused and Bounded, Risk-based, Evidence-Based, Transparent, and Total Product Lifecycle Perspective — and the document states that these five draw in particular on principle 10, the monitoring-and-retraining principle this lesson opened with. The first of the five turns the last step above from a recommendation into a requirement. A focused and bounded plan means “plans in place to safely modify the device within the bounds of the PCCP, including methods for verifying and validating the changes and mechanisms to detect and revert or stop implementation of a change that fails to meet specified performance criteria” — Guiding Principle 1, “Focused and Bounded”.

Progressive exposure and rollback are the engineering names for a detect-and-revert mechanism. The regulators' formulation adds the part teams skip. The mechanism has to be part of the plan, in advance. Otherwise the change was never bounded in the first place.

The smallest adequate intervention can reduce risk and cost, and the four are not interchangeable. Retraining updates learned parameters from a new training dataset. It addresses representation or conditional changes, requires full qualification, and can introduce model churn. Recalibration adjusts the probability mapping while preserving ranking behavior. It targets calibration drift, needs recent representative labels, is usually cheaper, and does not fix missing features. A threshold or policy update changes the action rule applied to model outputs. It responds to cost or capacity changes, must be versioned independently, can alter fairness and workload, and does not improve the model score. Data or feature repair corrects source, label, or transformation defects. It may restore the current model, is often safer than retraining on bad data, and requires backfill and impact analysis.

The fourth column has the best-documented case. A commercial algorithm ranked patients for extra care, and at the 97th percentile of risk score Black patients had 26.3% more chronic illnesses than White patients — 4.8 versus 3.8 distinct conditions, P<0.001. Removing the disparity would raise the share of Black patients receiving extra help from 17.7% to 46.5%. That measurement appeared in Science on 25 October 2019. The defect was in what the model had been asked to predict. Obermeyer and three co-authors kept the same sample, the same predictors and the same training process, and changed the target: “Rather than future cost, we created an index variable that combined health prediction with cost prediction. This approach reduced the number of excess active chronic conditions in Blacks, conditional on risk score, to 7758, an 84% reduction in bias.”

No retraining schedule would have found that. Refitting parameters on the old label, nightly and forever, reproduces the bias every night with a fresh timestamp on it. And the cost of not intervening is on the record. The same day, the New York State Department of Financial Services and Department of Health wrote jointly to UnitedHealth Group about the same algorithm, Optum's Impact Pro.

A mature loop can conclude that the active model remains the safest option. Candidate generation is cheap. Production change, label feedback and operational instability are not.

Track rejected candidates and trigger history, so the team can distinguish genuine deterioration from noisy alarms. A loop with no rejections in its history has not demonstrated that it is careful. It has demonstrated that it cannot decline, which is the same thing as having no qualification stage at all. Detect-and-revert, in the sense the 2023 guiding principles require, presupposes that stopping a change is one of the outcomes the system knows how to produce.