Capstone: Design a Learning System and Defend Every Signal

A city receives resident reports, inspection records, sensor readings, weather data, repair histories, and contractor notes. Leaders want to identify emerging hazards, prioritize inspections, estimate repair duration, and allocate crews.

The request sounds like one AI project. It is not. Inside it sit prediction, ranking, anomaly detection, resource optimization, selective labeling, and causal policy questions. Your task is to separate them before proposing a system. Every failure this capstone asks you to guard against has already happened to a named city, and the numbers are on the record: a lead-pipe search in Flint, a violence-risk score in Chicago, a fraud system in Michigan, a flu tracker at Google, a court in The Hague.

The capstone begins with stakeholder and consequence mapping. Each row below is a group whose exposure to the same ranked list is different. In several cases the size of that difference has been measured.

Each component receives a different signal and requires a separate claim. Hazard triage is supervised classification or ranking from adjudicated inspection outcomes. Emerging pattern discovery is anomaly detection over sensor and repair signals. Repair duration is regression with intervals and site-specific uncertainty. Inspection selection is active learning under inspector capacity and geographic coverage. Crew allocation is constrained optimization using predictions, skills, travel, and policy. Policy impact is causal evaluation of interventions, not ordinary prediction.

The inspection-selection contract is not an abstraction. A city has already signed it. In 2018 five researchers built an adaptive — that is, active-learning — model over 55,893 Flint properties, to decide which homes to excavate in the search for hazardous service lines. Their paper states what it could do: “we emphasize that our predictive model is empirically accurate for estimating whether a Flint home’s pipes are safe/unsafe, with an AUROC score of nearly 0.92, and a true positive rate of 97%”.

The number that matters for a capstone is not the AUROC. It is what the selection policy bought. Backtested on the 6,505 homes the FAST Start program actually visited, the model cut the rate of unnecessary service-line replacement visits from 18.8% to 2.0%. Across 18,000 planned replacements that is worth roughly $11M — about 2,100 more homes made safe on the same budget. Active learning under capacity is not a technique on a diagram. It is 2,100 households.

The capstone rewards a justified scope rather than the largest architecture. A decision-support pilot predicts and ranks cases while inspectors retain final authority: lowest interaction risk, creates reviewed feedback, supports shadow evaluation, may improve workflow slowly. A hybrid operations system combines rules, predictive models, anomaly alerts, and optimization: it addresses several tasks and keeps constraints explicit, but it requires interface testing and both component and system metrics. A research-only phase improves labels, reporting coverage, and causal evidence before deployment: appropriate when targets are weak, it reduces premature automation and can test data interventions, and it requires organizational patience.

The mildest phrase in that first column — may improve workflow slowly — is the one that needs evidence. The same architecture has been measured, and it went the other way. Computer-aided detection for screening mammography is the decision-support pilot in its purest form: the system flags and ranks, the expert keeps final authority. It was FDA-cleared, Medicare-reimbursed and widely adopted. Then Fenton and colleagues published the numbers in the New England Journal of Medicine in 2007. They studied 429,345 mammograms from 222,135 women across 43 facilities. After CAD was introduced, specificity fell from 90.2% to 87.2% and biopsies rose 19.7%.

Eight years later Lehman and colleagues looked again, across 323,973 women, and found no improvement on any performance measure. Sensitivity was actually lower among the radiologists who read both with and without it (odds ratio 0.53, 95% CI 0.29–0.97). Their conclusion in JAMA Internal Medicine was blunt: “Computer-aided detection does not improve diagnostic accuracy of mammography. These results suggest that insurers pay more for CAD with no established benefit to women.”

Two large studies, eight years apart, on a system that left the human in charge throughout. Human authority is not a safety guarantee. A pilot whose evaluation plan cannot return that verdict is not an evaluation plan.

Complete each step and preserve evidence for reviewers. Frame the decisions: define users, prediction moments, actions, and costs for every component. Audit feedback: trace labels, reports, outcomes, demonstrations, and missing counterfactuals. Choose approaches: justify supervised, anomaly, active, optimization, or causal methods separately. Establish baselines: compare current policy, transparent rules, and simple models. Design evaluation: specify protected time and geography splits, queues, intervals, slices, and experiments. Govern deployment: set review authority, fallback, appeal, monitoring, and rollback. State stop conditions: list the evidence that would pause, narrow, or cancel the project.

Steps six and seven are the ones students treat as paperwork. Flint is the reason they are not. In 2018 a new program manager stopped using the model's rankings. The consequence was quantified in a federal court filing rather than a paper. For 1 January–15 August 2018 the city's observed hit rate was 19.7%. A random dig would have produced 31.4%. The city made 163 digs, 4.5% of the total, in Ward 5 — the ward the model ranked highest, where the hit rate was 95.7%. It made 702 digs, 18.6%, in Ward 4, the ward the model ranked lowest, where the hit rate was 2.4%. Eric Schwartz, one of the model's builders, swore to that analysis and summed it up in a sentence: “The results of my analysis show that, at an aggregate level, the City is not conducting excavations at homes that are most likely to have a hazardous (lead or galvanized steel) service line.”

Nothing had gone wrong with the modelling. The handover to a contractor is where the value was lost — step six. And the statistic that exposed it, hit rate by ward compared against predicted rank, is precisely the monitoring metric step seven exists to write down in advance. A memo that specifies that comparison would have caught the failure in weeks instead of a litigation cycle.

A reviewer should be able to reconstruct the reasoning without guessing. Of the items below, the human-roles diagram is the one written last and the one that matters first. There is a measured price for leaving it out.

Michigan's MiDAS unemployment system issued fraud determinations with no human adjudicator. It then terminated benefits, garnished wages and intercepted tax refunds on the strength of them. In 2019, in Cahoo v. SAS Analytics Inc., the U.S. Court of Appeals for the Sixth Circuit set out what a review had found: “this system was deeply flawed; the Michigan Auditor General reviewed over 22,000 of MiDAS’ fraud determinations and found that 93% of them did not actually involve fraud. In other words, 93% of MiDAS’ fraud adjudications were false-positives.” The Michigan Supreme Court repeated the 93% figure in 2022, in Bauserman v. Unemployment Insurance Agency, and held that the claimants had stated a claim on which they may recover money damages for the due-process violation. The authority-and-appeal layer is the difference between a false-positive rate and a constitutional injury.

A strong capstone does not hide uncertainty behind a sophisticated method. It names which claims are predictive, causal, exploratory, or policy-driven, and it attaches an appropriate evidence plan to each of them.

Chicago shows how long a system can run while its central claim goes untested. CPD spent $3.8 million in federal grants across six versions of its "party to violence" risk models. As of July 2018, 399,412 individuals held an SSL risk score. As of March 2019, 313,513 held a CVRM risk tier. The causal question — does scoring these people reduce harm to them? — was examined by RAND, in a quasi-experimental evaluation of the 426-person 2013 pilot. It found no effect on victimisation: “Individuals on the SSL are not more or less likely to become a victim of a homicide or shooting than the comparison group, and this is further supported by city-level analysis.” In January 2020 the City of Chicago Office of Inspector General reported that the scores were unreliable and that the program had been decommissioned on 1 November 2019.

Ranking hundreds of thousands of people is a predictive claim. That the ranking helps them is a causal one, and no amount of the first ever becomes the second. If the data cannot support every component, the final recommendation should be narrower than the original request. Responsible scope is a technical achievement.