Interpreting, Naming, and Profiling Clusters

A commercial risk algorithm used across US health systems sorted patients by predicted health-care cost. The groups it produced were read as groups of high-need patients. One measurement profile, one human-facing name, and no step in between.

The dissection of that algorithm was published in Science on 25 October 2019, by Obermeyer and three colleagues. At a given risk score, they found, Black patients were considerably sicker than White patients, because less money had historically been spent on their care. The name was a judgement about need. The numbers underneath it were a record of spending.

The size of the gap between the two is in the abstract: “Remedying this disparity would increase the percentage of Black patients receiving additional help from 17.7 to 46.5%.” Nothing in the grouping step was broken. The features were real, the scores were reproducible, and the failure lived entirely in the sentence that said what the group was.

On the same day, New York's Department of Financial Services and Department of Health wrote jointly to UnitedHealth Group's CEO about the Impact Pro program, asking that the algorithm be justified or withdrawn. Interpreting a cluster needs evidence, uncertainty, and language proportionate to the observed data. A regulator may arrive before the next retraining does.

Names should communicate observed evidence without implying unsupported causes or worth. Two arms of the US government have put on the record what the industry actually calls its segments, so the contrast does not have to be invented.

The Federal Trade Commission studied nine data brokers and reported in May 2014 that “For example, one of the nine data brokers has 3000 data segments for nearly every U.S. consumer.” Among the categories it found were “Urban Scramble” and “Mobile Mixers”, both covering high concentrations of Latino and African-American consumers with low incomes; “Rural Everlasting”, single people over 66 with “low educational attainment and low net worths”; and “Thrifty Elders”. The Senate Commerce Committee's staff report of 18 December 2013 listed products titled “Rural and Barely Making It”, “Ethnic Second-City Strugglers”, “Retiring on Empty: Singles”, “Tough Start: Young Single Parents” and “Credit Crunched: City Families”.

Bounded description. The FTC entry contains its own well-behaved alternative: single people over 66 with low educational attainment and low net worths. It states measurable characteristics. It names the source file and the window they were measured in, it can carry uncertainty, and it stops being true when the measurement stops being true. It is version-specific by construction.

Speculative interpretation. “Rural Everlasting”, “Ethnic Second-City Strugglers” and “Tough Start: Young Single Parents” do something the data cannot. They turn an association into a motive and a permanent condition, they stigmatize the people inside them, and they outlive the fit that produced them. A household leaves the age band. The file keeps the noun.

Operational code. While the evidence is incomplete there is a third option that commits to nothing: a neutral identifier such as “Regime B-04”, carried with an accompanying profile. It avoids premature narrative and supports controlled iteration. A code cannot be quoted back at a person in a decision meeting as though it were a finding.

A defensible description uses several views instead of one centroid table, and there is a demonstration of exactly how little a centre and a spread pin down. Matejka and Fitzmaurice, of Autodesk Research, opened their 2017 paper by stating the tradition they were extending: “Datasets which are identical over a number of statistical properties, yet produce dissimilar graphs, are frequently used to illustrate the importance of graphical representations when exploring data.”

What they built is the Datasaurus Dozen: thirteen two-dimensional datasets whose x mean (54.26), y mean (47.83), x standard deviation (16.76) and y standard deviation (26.93) agree to two decimal places. The point clouds form a star, a bullseye, parallel lines, a circle — and Alberto Cairo's original Datasaurus, the dinosaur that seeded the search. The datasauRus package on CRAN redistributes all thirteen and recomputes the summaries from the data itself: mean x 54.3, mean y 47.8, sd x 16.8, sd y 26.9, and a Pearson correlation of about −0.06 in every one. Thirteen populations, one profile row. A reader who has seen only the row knows nothing about which of the thirteen is on the table.

So the profile is built in layers. Membership and uncertainty: cluster size, soft assignment, boundary distance, and unstable members. Within-cluster distribution: ranges, quantiles, missingness, subgroups, and multimodality rather than only means. Contrasts: differences from the whole population and from the nearest competing cluster. Exemplars and counterexamples: representative, boundary, contradictory, and rare observations. Operational interpretation: the actions, risks, and limits that domain review or an experiment will support — and no more.

Useful reporting shows what the average conceals, and stability is one of the things it can show with a number rather than a feeling. Christian Hennig's cluster-wise bootstrap resamples the data. For each original cluster it records the mean Jaccard similarity to its closest counterpart in the resampled clustering. The interpretation scale is written down in the documentation for clusterboot() in his R package fpc: “Generally, a valid, stable cluster should yield a mean Jaccard similarity value of 0.75 or more.” Between 0.6 and 0.75 the cluster indicates a pattern in the data, but exactly which points belong to it is highly doubtful. Below a mean of 0.6, clusters should not be trusted. 0.85 and above is highly stable. A separate rule covers single runs: a Jaccard similarity of 0.5 or less in an individual resampling run counts that run as a dissolution of the cluster — the fpc default dissolution=0.5, a per-run criterion rather than a threshold on the mean.

The report should support understanding without overstating certainty. Start with neutral IDs, keeping technical cluster identifiers separate from human-facing names. Summarize distributions, reporting quantiles, missingness, variability, and subgroup composition in original units. Compare neighbors, contrasting each cluster with the full population and with its nearest alternative. Review examples, sampling central, boundary, unstable, rare, and contradictory members. Check consequences, assessing whether the proposed names or actions stigmatize, exclude, or create feedback loops. Then version the profile, tying every description to a dataset, feature pipeline, model, and time period.

The fifth step is the one with case law behind it. On 28 March 2019 the Department of Housing and Urban Development charged Facebook with housing discrimination over its targeted advertising and user-grouping practices. Secretary Ben Carson's line in that release was “Facebook is discriminating against people based upon who they are and where they live.” The Justice Department then sued Meta Platforms and settled in June 2022. Under the settlement Meta stopped using its “Special Ad Audience” tool for housing ads. It built a Variance Reduction System to reduce sex and estimated race/ethnicity disparities in who actually sees a housing ad. It paid $115,054, the maximum civil penalty available under the Fair Housing Act at the time.

Read that as a specification for step 5. A behaviourally derived audience that lines up with a protected characteristic, and is then acted on, has already been ruled on twice. The check is not whether anyone put a protected attribute into the feature matrix.

The clustering algorithm produces memberships under a geometry. Analysts then build a narrative from summaries and examples. Those two stages should not be conflated, and European law separates them at exactly the same seam.

The General Data Protection Regulation defines profiling, in Article 4(4), as automated processing of personal data used “to evaluate certain personal aspects relating to a natural person”. The Article 29 Data Protection Working Party reads that verb strictly. Its guidelines on automated individual decision-making and profiling, adopted in 2017 and revised in 2018, set out three elements of profiling and hold that “evaluating” implies “some form of assessment or judgement about a person”. Which is why the guidelines can also say that “A simple classification of individuals based on known characteristics such as their age, sex, and height does not necessarily lead to profiling.”

The same assignment, in other words, is or is not profiling depending on the purpose of the description built on top of it. Article 22(1) attaches a right not to be subject to solely automated decisions once that line is crossed. The geometry did not change; the second stage did. A good profile can be challenged, updated, and retired. It tells readers which observations support the description, which members do not fit it cleanly, and what the description is for.