Capstone: Design and Defend an ML Evidence Platform

A multinational marketplace wants one evidence platform. It has to carry account-takeover detection, seller-risk review, recommendations, and operational forecasting at once. That means batch training, low-latency features, delayed outcomes, streaming corrections, graph relationships, and regional privacy constraints on the same foundation.

You are not asked to choose one fashionable architecture. You must show which decisions share infrastructure, which semantics remain domain-owned, and how the system behaves when evidence is late, missing, disputed, poisoned, deleted, or too expensive to recompute.

Each of those failures has already happened to somebody, on the record, with a docket number. Four of them grade this capstone. Knight Capital Americas, on 1 August 2012. The 2017 Equifax breach, as the Government Accountability Office reconstructed it. The Horizon accounting records examined in Bates & Others v Post Office Ltd. And SyRI, the Dutch fraud-scoring system struck down by the District Court of The Hague on 5 February 2020.

Strong architectures show the interfaces and guarantees between five planes. The decision plane names the request, latency, subject, action, threshold owner, and fallback. The evidence plane holds events, dimensions, labels, identities, clocks, exposure, and permitted history. The pipeline plane covers ingestion, storage, joins, transformations, materialization, and snapshots. The assurance plane carries contracts, validation, temporal tests, parity, observability, incidents, and rollback. The governance plane governs purpose, minimization, access, retention, deletion, documentation, and approvals.

The failures later in this lesson are each a break in one plane that arrived through another. A deployment procedure that did not exist cost Knight Capital $460 million in about 45 minutes; that is the assurance plane. A certificate that had expired roughly 10 months earlier hid an intrusion at Equifax; assurance again. Legislation that could not be inspected ended SyRI; that one is governance. None of the three shows up in a training table.

Choose the smallest design that satisfies the decision and recovery requirements. A nightly batch system is simple and reproducible. It remains the right home for labels, long-history features, and backfills, and it cannot meet rapidly changing device-risk needs. A streaming-only system gives low latency for recent behavior, then makes replay, correction, and historical joining hard; reference data and labels still need batch paths. A hybrid evidence system uses event streams for fresh state and batch snapshots for history, labels, and reconciliation. It fits mixed latency and correction needs. It also adds coordination between the two paths, and that coordination has to be tested rather than assumed.

Do this before drawing technology boxes. Define the example grain and a unique prediction key. Fix the event, server, and availability clocks used at login or payment time. Separate confirmed takeover from confirmed legitimate, and both from unknown and censored cases. Record which cases received review, and which model score influenced that review. Keep accounts and linked identities together in the split.

Step 3 is the one teams treat as a free choice. For a consumer account-takeover or fraud platform it is not one. The label-arrival clock is written into Regulation E. Under 12 CFR § 1005.11 the consumer has until 60 days after the institution sends the periodic statement to give notice of an error. From that notice the rule runs: “A financial institution shall investigate promptly and, except as otherwise provided in this paragraph (c), shall determine whether an error occurred within 10 business days of receiving a notice of error.” The institution may take up to 45 days instead, if it provisionally credits the account within 10 business days.

The extensions are asymmetric, and the split protocol has to encode them. Those windows become 20 business days and 90 days for transfers not initiated within a state, point-of-sale debit card transactions, and accounts opened within the previous 30 days. Bank examiners are told the same thing. The Federal Deposit Insurance Corporation states it in its examination manual: notice “not later than 60 days after sending a periodic statement”, and “the time periods are extended from 10 and 45 days, to 20 and 90 days, respectively”. So a training table cut today holds a knowable population of cases that are still legally open. It also holds a new-account cohort whose labels mature on a different schedule from everyone else's.

Every feature must have a source, owner, timestamp, and online availability story. Inventory the events and dimensions with their authority, contracts, coverage, corrections, and privacy classification. Resolve identities temporally, without rewriting historical ownership. Separate streaming state for recent events from batch sources for stable long history. Reconstruct only values that were available before each historical scoring request. Then specify missing-device, stale-cache, cold-start, and source-outage behavior.

Banking supervisors wrote this expectation down in January 2013. The Basel Committee asks that risk data “be aggregated on a largely automated basis so as to minimise the probability of errors”. A bank “should strive towards a single authoritative source for risk data per each type of risk”. Its completeness principle asks a bank to “capture and aggregate all material risk data across the banking group”. One source per risk, assembled by machine, with nothing material left out. That is a source inventory, written by a regulator.

The architecture should support safe retry, backfill, comparison, and rollback. Preserve raw evidence and immutable training releases. Make jobs idempotent, with stable keys and atomic publication boundaries. Version the online and offline materialization. Keep manifests linking sources to consumers. Document the recovery path.

Step 5 is where an architecture is actually tested, and Knight Capital Americas is the case to design against. On 1 August 2012 a staged manual deployment skipped one of eight SMARS routing servers, leaving obsolete “Power Peg” code active there. In about 45 minutes the firm took 4 million executions in 154 stocks for more than 397 million shares, and realised a $460 million loss. The instructive part is not the bad release. It is the recovery attempt, which the SEC set out like this: “In one of its attempts to address the problem, Knight uninstalled the new RLP code from the seven servers where it had been deployed correctly. This action worsened the problem, causing additional incoming parent orders to activate the Power Peg code that was present on those servers, similar to what had already occurred on the eighth server.”

The SEC's order came on 16 October 2013. It fined Knight $12,000,000, and found that the firm had no written code deployment procedures and no requirement for a second technician to review a deployment. The Federal Reserve Bank of Chicago described the same morning from outside the firm: “On Wednesday, August 1, 2012, a $440 million loss in 45 minutes brought market maker Knight Capital to the brink of bankruptcy”. Its author, Carol Clark, listed the controls that were absent — order-rate limits, a kill switch, intraday position limits and profit-and-loss limits. Your recovery plan must therefore state, before an incident, which action contains and which action spreads. And no partial rollout may reach production without a second reviewer recorded in the manifest.

The release plan should include concrete controls with owners and responses. Two of these gates have documented body counts, and the bullets name them.

Extend the architecture beyond quality and throughput. Threat-model the sources. Define the privacy flows. Publish the documentation. Design retirement.

Some of this is now a legal requirement. Article 10(3) of the EU AI Act covers the training, validation and testing data sets of high-risk systems. They “shall be relevant, sufficiently representative, and to the best extent possible, free of errors”. The same paragraph asks for “the appropriate statistical properties”, including for the groups affected. Representativeness is judged against the intended purpose, never in the abstract.

Step 1 has a price list. Nicholas Carlini and eight colleagues showed how cheap it is to poison a slice of a web-scale training set. They presented the work at the IEEE Symposium on Security and Privacy in 2024: “By exploiting specific invalid trust assumptions, we show how we could have poisoned 0.01% of the LAION-400M or COYO-700M datasets for just $60 USD.” NIST's adversarial machine learning taxonomy, published in March 2025, cites that work and names the countermeasure: “For preventing data poisoning with web-scale data dependencies, this includes verifying web downloads as a basic integrity check to ensure that domain hijacking has not injected new sources of data into the training dataset”. Poisoning is not a line on a risk register, then. It is an acceptance test: every externally fetched source verified against a hash published by the publisher.

The same step covers controls that fail silently rather than loudly. Attackers reached the personal information of at least 145.5 million individuals through Equifax's online dispute portal — the finding of a Government Accountability Office report published on 30 August 2018. The Federal Trade Commission settled on 22 July 2019, requiring Equifax to pay at least $575 million. The settlement alleged that “Equifax failed to patch its network after being alerted in March 2017 to a critical security vulnerability affecting its ACIS database”. The company's own security team had ordered vulnerable systems patched within 48 hours. A control that exists on paper and sees nothing for ten months is worse than a missing one. It gets credited in the design review.

Step 2 has a stop condition with a date on it. On 5 February 2020 the District Court of The Hague ruled on SyRI (Systeem Risico Indicatie), a system that linked government databases to score citizens for benefit and tax fraud. The court held that the enabling legislation did not strike the “fair balance” required by Article 8 ECHR. It declared Section 65 of the SUWI Act and Chapter 5a of the SUWI Decree to have no binding effect. A privacy flow that cannot be inspected by the people scored through it has already been held unlawful once — for a platform doing the work this capstone describes.

Decide which mechanisms should be shared and which remain domain-specific. Model the workload first: scan, shuffle, state, online serving, labeling, and backfill demand. Compare batch, streaming, feature-store, and hybrid options under failure, not under happy-path throughput. Then price the whole lifecycle, including migration, incidents, retention, recovery, and human review. Knight's $460 million in about 45 minutes and Equifax's settlement of at least $575 million are lifecycle costs of the assurance and governance planes. They are not line items outside the platform budget. Last, preserve reversibility: state the portability, exit, fallback, and rollback conditions before the first dependency is created.

Produce an example and label specification, source inventory, event and data contracts, architecture diagram, temporal-feature specification, and split protocol.

Add a dataset release manifest, feature registry entries, validation matrix, service objectives, incident runbook, backfill plan, and privacy datasheet.

Include three adversarial timelines that prove future values are excluded. Include three source failures, and show containment, fallback, and recovery for each. Make one of them the Knight sequence, in which the rollback itself is the accelerant. Then the runbook has to say which action contains and which spreads.

Finish with a decision record. It should explain why the chosen architecture is sufficient, which assumptions remain uncertain, and which measurements will decide where the next money goes.

Ask whether the design can answer one prediction end to end. Which raw events produced each feature? Which versions and clocks were used? What happens when one source is absent?

Ask whether offline metrics could still be inflated. Test reviewed-case selection, identity overlap, future corrections, current-state joins, split contamination, and synthetic derivatives.

Ask whether operations are recoverable. The system should survive duplicate delivery, partial failure, stale materialization, a bad schema release, and a historical backfill. It should also survive the recovery action itself, which at Knight Capital removed the new code from seven correctly deployed servers and made the loss larger.

Finally, ask whether the dataset should exist in its proposed form. Verify purpose, minimization, access, retention, deletion, and documentation before anyone builds it and creates irreversible data dependencies. On 5 February 2020 the District Court of The Hague answered that question for a system already in production: “The court holds that the legislation pertaining to the application of SyRI is insufficiently transparent and verifiable.” Section 65 of the SUWI Act and Chapter 5a of the SUWI Decree were declared to have no binding effect. The stop condition is real, it has a citation, and it is cheaper to apply before the platform is built.

The memo should name the decision contracts, source authorities, temporal rules, identity model, label maturity, data releases, online evidence path, quality gates, SLIs, security controls, governance conditions, and cost model.

Include rejected alternatives, unresolved risks, responsible owners, staged rollout, stop conditions, and the evidence that would justify redesign. A valid conclusion may be to avoid real-time processing altogether, or to keep one domain outside the shared platform.

Write the stop conditions in the form the record supports. What would make this platform unlawful to inspect, as SyRI was held to be on 5 February 2020? What would make its records unusable as evidence, as Legacy Horizon was found to be? And who is required to approve a partial deployment — which at Knight was nobody?