Source Discovery, Ownership, and Data Authority

None of these examples necessarily triggers a database type error.

A column can keep the same name and type while its meaning changes. A status field may be redefined, a timestamp may switch time zones, an identifier may be recycled after account deletion.

The cost of one redefined flag is on the public record. On 1 August 2012 Knight Capital Americas lost over $460 million in 45 minutes. The wire format of the flag that did it never changed. The U.S. Securities and Exchange Commission's settled order found the mechanism: “The new RLP code also repurposed a flag that was formerly used to activate the Power Peg code.” Seven of eight servers carried the new Retail Liquidity Program code. On the last one, “orders sent with the repurposed flag to the eighth server triggered the defective Power Peg code still present on that server”. What followed was fast. “SMARS routed millions of orders into the market over a 45-minute period, and obtained over 4 million executions in 154 stocks for more than 397 million shares”, and “Knight lost over $460 million from these unwanted positions”. The Commission imposed “a civil money penalty in the amount of $12,000,000”. Carol Clark, writing for the Federal Reserve Bank of Chicago, described the same day from outside the firm, and with a different figure for the loss: “On Wednesday, August 1, 2012, a $440 million loss in 45 minutes brought market maker Knight Capital to the brink of bankruptcy.” One value; two meanings; two populations of machines.

ML pipelines are unusually sensitive to these semantic shifts, because historical patterns are compared across versions. A change that is acceptable for an application screen can invalidate a feature or a label.

A data contract makes the producer–consumer relationship explicit. It records the structure, meaning, allowed values, update behavior, ownership, privacy classification, and compatibility rules of a data product.

Contracts do not prevent every failure. They create a place where assumptions can be reviewed, tested, versioned, and negotiated before a silent change reaches training or serving.

There is a proposed shape for that record, and it borrows from electronics. Every component there, however simple, ships with its operating characteristics and recommended uses. Every dataset should arrive the same way, with a sheet documenting “its motivation, composition, collection process, recommended uses, and so on”. That is the argument Timnit Gebru and her co-authors made in Communications of the ACM in December 2021. Their opening complaint still holds: the community has “no standardized process for documenting datasets”.

A source may be authoritative for one question and unsuitable for another. A system of record owns the business state under a declared transaction and correction process. A system of capture records events near their origin, and needs identity and clock contracts before anything can be joined to it. An analytical replica is optimized for access, and inherits its authority from lineage and policy rather than from the data itself.

How thin an identifier's authority can be is measurable, and telephone numbering is the documented case. In 2018 the Federal Communications Commission put a number on it in its order on unlawful robocalls: “Approximately 35 million numbers are disconnected and made available for reassignment to new consumers each year”. The same order establishes the Reassigned Numbers Database and fixes the interval before reuse: “We establish a minimum aging period of 45 days, in the middle of the proposed range, for all numbers.” Reassignment is not an error in the system of record. It is the system of record working as designed.

How often the recycled key still points at the previous holder has been counted. Kevin Lee and Arvind Narayanan reported the count in 2021: “We sampled 259 phone numbers available to new subscribers at two major carriers, and found that 171 of them were tied to existing accounts at popular websites”. They also found that “a significant fraction (100 of 259) of the numbers were linked to leaked login credentials on the web”. The same two authors give the broader figure on Freedom to Tinker, published by the Princeton Center for Information Technology Policy: of those 259 numbers, “215 of them were recycled and vulnerable to either account hijackings or PII indexing”. A pipeline that keys a customer on a phone number is not making a type error when it merges two people. It is trusting an identifier whose owner has published the rate at which it is handed to somebody else.

An ML team needs operational and semantic metadata for every critical dependency: which system holds authority, what each field means, when a value is generated, corrected or backfilled, who owns the incident, and which consent, retention and residency obligations apply.

The clock belongs on that inventory. It has a producer, a schedule, and a release note six months ahead of the change. A positive leap second was inserted between 23:59:59 UTC on 31 December 2016 and 00:00:00 UTC on 1 January 2017, displayed as 23:59:60. Cloudflare's own postmortem opens on what that did inside its infrastructure: “At midnight UTC on New Year's Day, deep inside Cloudflare's custom RRDNS software, a number went negative when it should always have been, at worst, zero.” The negative duration reached a function that does not accept one — “rand.Int63n promptly panics if its argument is negative”. The blast radius is in the same post: “At peak approximately 0.2% of DNS queries to Cloudflare were affected and less than 1% of all HTTP requests to Cloudflare encountered an error”, and “The most affected machines were patched in 90 minutes and the fix was rolled out worldwide by 0645 UTC.” Thomas Claburn of The Register reported the incident independently, quoting Cloudflare's John Graham-Cumming — “We didn't plan in the code that the time difference would be negative” — and put the affected share at about 0.2 per cent of DNS queries across 102 data centres.

Nothing in that failure was a schema violation. The clock was a source dependency with a contract of its own, and the contract said that one minute of that year would contain sixty-one seconds.

A safe migration often publishes old and new versions together. Downstream teams can compare row counts, distributions, joins, features, and model outputs before switching. Backfills require special care, because new logic may reinterpret years of history. A model trained on rewritten past data can look stable offline while serving on records produced by the new live process. Record the effective date of the semantic change, and the transformation version used for backfilled rows. When old meaning cannot be reconstructed, document the discontinuity instead of hiding it.

The U.S. Bureau of Labor Statistics runs that rewrite on an announced schedule, and publishes its size. With the release of January 2026 data on 11 February 2026 it introduced the annual benchmark revision to the Current Employment Statistics survey of nonfarm establishments. The revision realigns that sample-based series to the administrative counts of the Quarterly Census of Employment and Wages. The discontinuity is stated by the producer itself: “Compared with the sample-based, seasonally adjusted published estimate for March 2025, total nonfarm employment had a revision of −898,000 or −0.6 percent.” The benchmarked March 2025 level is 158,377,000. Roberto B. Pinheiro and Rory G. Quinlan, writing for the Federal Reserve Bank of Cleveland, describe the mechanism as intended — “annual benchmarks are constructed to realign the sample-based employment estimates for March of each year with the universe employment counts for that month” — and give the span each revision touches: “Each benchmark revision affects 21 months of nonseasonally adjusted data.” A feature computed from employment levels on 10 February 2026, and the same feature computed on 12 February 2026, are drawing on different numbers for the same months. Same authoritative producer, no defect on either side.

Retire the previous version only after named consumers have migrated and rollback evidence exists. Compatibility is an operational process, not a boolean flag.

Choose one high-impact field. Establish who controls its meaning, and who decides when a value is corrected. Name the business concept apart from the table currently carrying it. Identify every producer that can create or alter it. Document which system decides current value, historical value and corrections. Trace the delay and filtering in each downstream replica. Then register the consumers.

The last step is the one teams skip, and the failure it prevents has a name. D. Sculley and nine colleagues gave it a section of its own in their 2015 paper on hidden technical debt in machine learning systems: “Without access controls, some of these consumers may be undeclared, silently using the output of a given model as an input to another system.” Their verdict on the arrangement is unambiguous — “Undeclared consumers are expensive at best and dangerous at worst, because they create a hidden tight coupling”.

It is not one team's anecdote. In 2021 Justus Bogner and two co-authors surveyed 21 primary studies for a systematic mapping study, and catalogued 72 antipatterns in AI-based systems. Undeclared consumers is one of them, recurring in three of the studies they reviewed: “An example related to modularity is undeclared consumers [P3,P11,P20], i.e. if the results of an AI component serve as input for a broad range of other systems or components. These undeclared or silent consumers constitute hidden coupling”. A register of consumers is what converts that hidden coupling into a list of people you have to call before you change the field.