Group Performance and Intersectional Evaluation

Disaggregated evaluation asks how error and service outcomes vary across groups, intersections, and operating contexts. A report should pair group averages with denominators, uncertainty, threshold, prevalence, and consequence. Take the denominator first. A nine-point gap across four thousand cases and the same gap across eleven are different findings. Only one of them supports a decision.

United States employment law has carried a numbered answer to that problem since 1978. The four-fifths rule sets a selection-rate ratio of 80% as the general threshold of evidence for adverse impact. It sits in the EEOC's Uniform Guidelines on Employee Selection Procedures. The same paragraph then qualifies that threshold in both directions, on sample size. Greater differences may not constitute adverse impact where they are “based on small numbers and are not statistically significant”. And where the thin numbers point the other way, the regulation does not stop at silence. It sends the team looking: “Where the user's evidence concerning the impact of a selection procedure indicates adverse impact but is based upon numbers which are too small to be reliable, evidence concerning the impact of the procedure over a longer period of time and/or evidence concerning the impact which the selection procedure had when used in the same manner in similar circumstances elsewhere may be considered in determining adverse impact.”

Read that as an instruction rather than a caveat. It does not tell a team to publish the thin cell. It does not tell them to drop it. It tells them to go and find a denominator — over a longer period, or from the same procedure used the same way somewhere else. Intersectional analysis can reveal harms concealed by broad categories. But sparse cells create unstable estimates and privacy risk. So the answer is rarely one table. It is quantitative slices, targeted data collection, qualitative evidence, and hierarchical or pooled methods where they fit.

One aggregate metric summarizes. Predefined slices test what somebody thought to ask about. Exploratory search finds the pocket nobody predicted. It also creates, in the same motion, the problem that keeps a striking pocket from counting as a finding.

Search a large number of candidate slices and the arithmetic turns against you. The field's own tooling names that problem and pays for it. Slice Finder automates the search, and its authors put the difficulty plainly in 2019: “One problem with performing many statistical tests (due to a large number of candidate slices) is an increased number of false positives. This is what is also known as Multiple Comparisons Problem (MCP)”. So the tool does not report whichever slice looked worst. It controls false positives with alpha-investing, which bounds the marginal false discovery rate E(V)/E(R) at level alpha. A search that skips that step has not found a disparity. It has drawn a hypothesis out of a hat, in a room full of hats.

Population dimension, task condition, metric, uncertainty, consequence: a slice result means very little until all five are stated together. One short paper carries all five in a single sentence. That is why it is worth reading as a template rather than as a medical curiosity.

A pulse oximeter clips to a finger and reports blood oxygen. In 2020 the New England Journal of Medicine published what happens when that reading is checked against the arterial blood gas underneath it. Sjoding and colleagues took the patients whose clip was showing a reassuring number: “In the University of Michigan cohort, among the patients who had an oxygen saturation of 92 to 96% on pulse oximetry, an arterial oxygen saturation of less than 88% was found in 88 of 749 arterial blood gas measurements in Black patients (11.7%; 95% confidence interval [CI], 8.5 to 16.0) and in 99 of 2778 measurements in White patients (3.6%; 95% CI, 2.7 to 4.7).” In a second, multicentre cohort the same comparison ran 160 of 939 measurements against 546 of 8,795 — 17.0% (95% CI 12.2–23.3) against 6.2% (95% CI 5.4–7.1).

Now lay the five elements over it. The population dimension is self-reported race. The task condition is the device reading 92–96%, not the device in general; the finding lives inside that band. The error metric is occult hypoxemia — an error in one specific direction, not an average deviation. The uncertainty is stated as raw counts and 95% confidence intervals. So a reader can see that the Michigan Black-patient estimate rests on 749 measurements and the White-patient estimate on 2,778. The consequence is the one that makes the rest matter: hypoxemia the clinician never sees, on a reading that told them to look elsewhere. Strip any one of the five and the result stops being actionable.

Slices chosen after the results arrive prove very little. Choosing them from harm pathways first is what makes the third step worth reporting.

Step one does not have to be invented from scratch. It has had a citable format since 2019. Nine authors proposed that a trained model ship with a short document recording how it was evaluated, and called it a model card. The abstract already specifies what a slice plan should contain: “Model cards are short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups (e.g., race, geographic location, sex, Fitzpatrick skin type) and intersectional groups (e.g., age and race, or sex and Fitzpatrick skin type) that are relevant to the intended application domains.”

Two things in that sentence do the work. The intersectional groups are named as examples, not gestured at — age and race, or sex and Fitzpatrick skin type. And relevance is tied to the intended application domain, which is what stops a slice plan from becoming an exhaustive cross-product nobody can power. Steps two through five then have something fixed to report against.

An overall accuracy figure is the wrong summary, and the largest demographic measurement of face recognition to date shows why. In December 2019 NIST ran 18.27 million images of 8.49 million people through 189 algorithms from 99 developers.

The headline is not that the systems are inaccurate. It is that the two error directions do not move together. False positive rates across demographics often vary by factors of 10 to beyond 100 times. False negatives usually vary by factors below 3. On sex, the report is explicit: “We found false positives to be higher in women than men, and this is consistent across algorithms and datasets. This effect is smaller than that due to race.” False positives were elevated in the elderly and in children too.

Declare the slices in advance. Then decide what a cell holding eleven cases is allowed to claim. New York City has written exactly that pair of artefacts into law. Under the 2023 bias-audit rule implementing Local Law 144, an audit of an automated employment decision tool must compute selection rates and impact ratios separately for sex categories, for race/ethnicity categories, and for intersectional categories of sex, ethnicity and race. It must publish the number of applicants in each category. It must state how many assessed individuals fell into an unknown category.

The sparse-cell rule is written down too, with a threshold and a price: “an independent auditor may exclude a category that represents less than 2% of the data being used for the bias audit from the required calculations for impact ratio. Where such a category is excluded, the summary of results must include the independent auditor's justification for the exclusion, as well as the number of applicants and scoring rate or selection rate for the excluded category.” The small cell may leave the impact-ratio calculation. It may not leave the page.

A written plan still has to be checked. That is the last artefact. The New York State Comptroller audited the city's enforcement of Local Law 144 over July 2023–June 2025. The city's Department of Consumer and Worker Protection had reviewed 32 companies and found one non-compliance issue. The auditors found at least 17 instances of potential non-compliance.

Averages conceal the groups a system fails. An evaluation that reports only the average has answered a question nobody asked. NIST needed 18.27 million images to establish that false positives and false negatives move by different orders of magnitude across demographics. Sjoding and colleagues needed the arterial blood gas standing behind the oximeter reading to separate 11.7% from 3.6%. Buolamwini and Gebru needed skin type crossed with gender to find 34.7 per cent. In none of these cases did the disparity appear in the number the system reported about itself.

Define when disaggregated and intersectional evaluation requires the team to redesign, restrict, remedy, or retire the system. And set the floor below that. In Bridges the Court of Appeal found the breach of the section 149 duty in the verification never sought, not in a table that came out badly.