Production Data Quality and Training–Serving Skew

A categorical feature starts reporting one of its values far more often than it used to. Google's data validation system in TFX watches exactly that, batch to batch, and the worked example its authors give for the alert is not a hypothetical: “if the highest change in frequency is observed in value '-1' then this might be an issue with some backend failing and producing a default value. (This is an actual situation observed in one of our production pipelines.)”

Nothing is missing in that pipeline. The feature is present, typed, in range, and populated on every request. Only its meaning has changed: from a measurement to a receipt for a failed lookup. A monitor watching nulls sees a clean field. A model that learned what '-1' meant in training now reads a growing cohort of lookup failures as a real population.

NASA lost the Mars Climate Orbiter on 23 September 1999 to a number that was present, well-formed, and inside its expected range. The ground software file SM_FORCES expressed thruster performance data in English units. The Angular Momentum Desaturation file that software produced was specified in metric units, and the trajectory modellers read it as metric, exactly as the interface documentation told them to. The mishap investigation board stated the root cause in November 1999 in one sentence: “The MCO MIB has determined that the root cause for the loss of the MCO spacecraft was the failure to use metric units in the coding of a ground software file, “Small Forces,” used in trajectory models.” At Mars insertion the spacecraft was approximately 170 kilometres lower than planned. Its development had cost $75 million, and it had launched on 11 December 1998.

No null check, no freshness check, and no range check anywhere in that chain would have fired. The producer wrote one unit. The consumer's contract declared another. Every value in between was valid. Quality checks that cover nulls and freshness miss semantic breakage. Production data quality is the degree to which inputs satisfy the meaning, timing, population, and compatibility required by the decision contract. A feature store that returns `account_age = 12` with a current timestamp has satisfied none of that by returning it.

Quality failures can preserve shape while breaking meaning, and each layer of the stack fails in its own way. Coverage is the layer that is hardest to see. Rows that never arrive leave no trace in the rows that did.

Between 25 September and 2 October 2020, 15,841 positive COVID-19 cases never reached the UK's reported daily figures. Public Health England announced the gap on 4 October and put it down to a fault in the data-load process into the reporting dashboards. The peer-reviewed analysis of the episode is more specific: case rows were truncated once a spreadsheet row limit was reached. Fetzer and Graeber put the mechanism in one sentence: “Because of a coding error involving spreadsheet data used by the health authorities, a total of 15,841 COVID-19 cases (around 20% of all cases) failed to have timely contact tracing.” They estimate between 126,836 and 185,188 additional reported infections and between 1,521 and 2,049 additional reported deaths over the following six weeks.

Every figure that did arrive was well-formed. The silent exclusion was the whole failure, and its cost was measurable only afterwards.

Training–serving skew occurs when logically equivalent examples receive different values or transformations across environments. Google's data validation team ranked the causes rather than listing them: “One of the issues that frequently occurs in production pipelines is skew between training and serving data. Several factors contribute to this issue but the most common is different code paths used for generation of training and serving data”. Their taxonomy splits the problem three ways: feature skew, distribution skew, and scoring/serving skew.

The rates are measured rather than estimated. Over a 30-day window in production, a "new feature column" anomaly fired in 10% of evaluation pipelines and out-of-domain categorical values in 6%. More than 70% of pipelines had at least one model unit test. Those tests were executed more than 80K times in a month, and 6% of the executions failed. Duplicated code, different join cutoffs, unavailable online fields, inconsistent defaults, and stateful transformations are all instances of the same shape.

Population drift is different. The pipeline may be consistent while the real world changes. Diagnose parity before treating a distribution shift as a modeling problem.

The containment action depends on the cause, and the two hardest to tell apart are a semantic contract change and a legitimate population shift. Google Flu Trends separates them. The model had been running high for two years before the count appeared in print: “GFT also missed by a very large margin in the 2011–2012 flu season and has missed high for 100 out of 108 weeks starting with August 2011 (see the graph).” That is Lazer and three colleagues, writing in Science in 2014. They traced it to "algorithm dynamics": Google's own search blog reported 86 changes in June and July 2012 alone.

The flu had not changed its behaviour. The upstream provider had changed the data-generating process while the modelled world stayed where it was — a semantic contract change wearing the costume of a population shift. The model looked stable and was wrong.

The failure had already been found running in the other direction. A 2013 reassessment in PLOS Computational Biology reported that the original model missed the spring 2009 A/H1N1 pandemic wave, giving incidence estimates 30–40 fold lower than ILI surveillance, and greatly overestimated the 2012/2013 A/H3N2 season.

Move from contract to source before changing the model. In one regulated domain that is not house style but a written expectation. In October 2021 the U.S. FDA, Health Canada and the UK MHRA jointly identified 10 Good Machine Learning Practice guiding principles — non-binding guiding principles rather than regulation. Principle 10 reads: “Deployed Models Are Monitored for Performance and Re-training Risks Are Managed: Deployed models have the capability to be monitored in “real world” use with a focus on maintained or improved safety and performance. Additionally, when models are periodically or continually trained after deployment, there are appropriate controls in place to manage risks of overfitting, unintended bias, or degradation of the model (for example, dataset drift) that may impact the safety and performance of the model as it is used by the Human-AI team.” Principle 3 states that clinical study participants and data sets, including the training and test datasets, are representative of the intended patient population. That is the coverage layer, made a condition of the product rather than a dashboard.

For the parity step, Google publishes its own remedy as Rule #29 of its Rules of Machine Learning: log the features actually used at serving time, and train on that log. Two code paths cannot disagree if only one of them ever produced the training data.

Defaults, imputations, clipping, and category buckets can convert data failures into valid-looking numbers. Both hidden failures in this lesson have that shape. A backend fails and the feature arrives as '-1', so the null rate never moves and the anomaly is visible only as a change in how often one value appears. Case rows past a spreadsheet row limit were dropped, so the 15,841 missing positives left no mark on the daily figures that were published.

Capture raw missingness and transformation-path metadata. Operators can then tell a real zero from an unavailable value, and an alert can name a culprit value rather than a distance. The monitoring design should preserve enough evidence to explain one affected decision without logging every sensitive input.