Semantic Contracts and Business Invariants

In England's breast screening programme the physical record was never wrong. A Service Specification written in November 2013 set the upper age limit in a way the IT system then in use did not implement. The mismatch ran for years without producing a single malformed row. The independent review published in December 2018 put the cause in the document rather than in the code: “A new Service Specification was written to provide specific instructions against which the programme could be commissioned and quality assured, but it included a level of specificity which did not align with the IT system then in use, and was not consistently implemented by the breast screening units.”

Around 117,000 of the 196,000 women on the final affected list were there for that reason alone. They had not been invited up until their 71st birthday, as the specification required. The figure announced at the outset was 450,000; it was later revised to around 122,000 women offered an additional screen. Nothing about a row was malformed. The definition of who belonged in the population had stopped matching the system that produced it.

A physical schema describes names, types, nullability, and sometimes ranges. Those checks catch malformed records. They do not establish what a value means or when it is valid.

A semantic contract names the business concept, unit, population, clock, update policy, and known exceptions. It also states whether a value is observed, inferred, corrected, or provisional.

Machine learning depends on these distinctions because the same bytes can imply different evidence. A model cannot compensate for a field whose meaning drifts invisibly.

Amazon built a system for exactly this layer and published it in 2018. The abstract states the ambition without decoration: “Our system provides a declarative API, which combines common quality constraints with user-defined validation code, and thereby enables ‘unit tests’ for data.” The named constraints read like assertions on the table — isComplete, isUnique, isNonNegative, isInRange, isLessThan, satisfies, satisfiesIf, hasConsistentType — and each compiles down to an aggregation query that Spark can run at scale. This was not a whiteboard proposal. The evaluation ran on an internal product-catalogue sample of roughly 120 million records, about 50GB in parquet, on a five-worker c3.4xlarge Elastic MapReduce cluster running Spark 2.0.2. A rule written this way is reviewable. It fails loudly instead of drifting quietly.

Each layer blocks a different class of failure, and none is sufficient alone.

The physical schema describes how values are encoded and which records are structurally valid: field names and types, nullability and allowed values, serialization and compatibility. It is useful against malformed payloads. The semantic definition explains the real-world concept the field represents: its unit and population, its event and availability times, whether the value is observed or derived. It is useful against silent meaning changes. The operational promise defines how the producer will change, publish and support the data: freshness and correction policy, ownership and escalation, deprecation and migration windows. It is useful against unmanaged evolution.

The Mars Climate Orbiter is the middle layer failing while the outer one holds. The project's Software Interface Specification required the SM_FORCES output in the Angular Momentum Desaturation file to be in newton-seconds. It was delivered in pound-seconds, underestimating the trajectory effect by the 4.45 conversion factor. The investigation board's 1999 report drew precisely the distinction this section is about: “The SIS, which was not followed, defines both the format and units of the AMD file generated by ground-based computers.” One specification carried two layers. The format was honoured and the unit was not, and nothing that parsed the file could tell the difference.

The cost of the missing layer was arithmetic. Corrected after-the-fact navigation estimates put the initial periapsis at 57 km against a planned 226 km, with a minimum survivable altitude of 80 km. The U.S. General Accounting Office later compressed the whole episode into one line: “A NASA mishap investigation team determined that the loss was due to the mistaken use of English rather than metric units in the navigation software.”

Useful invariants express relationships that should remain true even when implementations change. Identity: an order belongs to one account at a given effective time, and identifiers cannot be reused across tenants. Temporal: a feature published for a scoring event cannot depend on observations that became available later. Accounting: line items reconcile with order totals under documented tax, return, and currency rules. Population: every emitted record belongs to the declared cohort and exclusion policy. Lifecycle: deletion, correction, and consent changes propagate to the stated downstream surfaces.

The temporal one is the invariant that has been counted. Sayash Kapoor and Arvind Narayanan surveyed the damage in Patterns in 2023 and gave the violation a definition: “Data leakage is a spurious relationship between the independent variables and the target variable that arises as an artifact of the data collection, sampling, or pre-processing strategy.” They found leakage reported in 22 review papers covering 17 scientific fields and collectively affecting 294 papers. Temporal leakage, in their taxonomy, is the case where the test set contains data from before the training set. That is the invariant above, broken, and published.

An independent team then measured what breaking it does. Rosenblatt and colleagues tested five forms of leakage across four datasets and three phenotypes, and reported in Nature Communications in February 2024 that “Leakage via feature selection and repeated subjects drastically inflates prediction performance, whereas other forms of leakage have minor effects.” Not every violation of a temporal invariant costs the same. That is why the invariant is worth writing as a testable statement rather than a warning.

The goal is not to fill a template. It is to expose assumptions that would otherwise remain distributed across code and memory.

Bring one concrete record and walk a real value from creation through training to serving. Name every clock: event time, ingestion time, availability time, correction time, label time. Write invariants that describe relationships which should survive implementation changes. Simulate a breaking change and ask which consumers fail if meaning, units, cardinality, or lateness moves. Agree on release mechanics: owners, migration evidence, deprecation windows, rollback conditions.

The fourth step is the one teams skip, and it is the one that catches the failures above. The November 2013 Service Specification was written so the programme could be commissioned and quality assured against it. Nobody diffed it against the system that had to implement it. The gap surfaced years later as around 117,000 women on an affected list. The Software Interface Specification did define the unit. The question never asked was what the consumer would do if the file arrived in pound-seconds instead of newton-seconds. In both cases the specification existed and was readable. What was missing was a rehearsal of the change against the people downstream of it.

For a material change, dual-run old and new definitions on representative traffic. Compare record counts, slice distributions, affected entities, labels, and downstream model behavior. A migration should identify consumers that cannot move immediately, the last date when old semantics remain available, and the conditions for rollback. Historical snapshots must preserve which definition each run used.

A version of this ran at national scale. ICD-10-CM became mandatory for U.S. claims on 1 October 2015. Months before that date, on 6 July 2015, the producer and the consumer's representative published the terms of the change together. CMS and the American Medical Association announced a documented tolerance window: “While diagnosis coding to the correct level of specificity is the goal for all claims, for 12 months after ICD-10 implementation, Medicare review contractors will not deny physician or other practitioner claims billed under the Part B physician fee schedule through either automated medical review or complex medical record review based solely on the specificity of the ICD-10 diagnosis code as long as the physician/practitioner used a valid code from the right family.” The scope was made explicit rather than left to interpretation. The American Medical Association spelled out that “A ‘family of codes’ is the ICD-10 three-character category.” And the cutover itself was not softened: a valid ICD-10 code was still required on every claim from day one.

That is what a migration looks like when it is written down instead of announced. A hard date for the new semantics. A bounded period in which one specific kind of imprecision is not penalised. A named scope, so both sides know what the tolerance does not cover. And both parties signing it. The decision to accept a change belongs to the consumer risk owner, not only the source team. A producer can certify that the code does what the contract says without knowing whether the new meaning is fit for a model.