Unsupervised Learning: Searching for Structure Without Task Labels

Removing task labels does not merely remove one dataset column. It changes what can be claimed.

The third question is the only one that can be settled with a number, and it has been. SimCLR learned image representations with no labels at all. Labels were brought back once, at the end, only to measure what the representation had preserved. The 2020 abstract reports what that measurement found: “A linear classifier trained on self-supervised representations learned by SimCLR achieves 76.5% top-1 accuracy, which is a 7% relative improvement over previous state-of-the-art, matching the performance of a supervised ResNet-50.” Fine-tuned on 1% of the labels, the same representations reached 85.8% top-5.

The downstream check then inherits the problem it was brought in to solve, because it is a choice too. A 2021 evaluation ran 13 self-supervised models across 40 downstream tasks and found the ranking moves with the task you pick. ImageNet top-1 correlates with many-shot linear transfer at Pearson r = 0.93. With semantic segmentation it correlates at r = 0.19. The same thirteen models, the same forty tasks: one downstream question makes the leaderboard look settled and another makes it nearly arbitrary.

Unsupervised learning uses observations that carry no target label defined for the immediate task. What it does with them varies. It may group examples, compress dimensions, estimate density, detect unusual points, or learn reusable representations.

The data still contain choices. Collection, features, preprocessing and boundaries are all decisions. Unlabeled does not mean neutral, natural, or free from human design.

How far from neutral has been measured. Off-the-shelf GloVe vectors are trained on the 840-billion-token Common Crawl corpus — raw web text, no labels anywhere in that pipeline. In 2017 Caliskan and two colleagues ran their Word Embedding Association Test over those vectors, in Science, and recovered every human bias they tested, at the sizes psychology had already recorded for people. “With 32 participants, the IAT comparing flowers and insects results in an effect size of 1.35 (p < 10−8). Applying our method, we observe the same expected association with an effect size of 1.50 (p < 10−7).” European-American versus African-American names came out at 1.41. Male-versus-female names paired with career-versus-family came out at 1.81. One figure should end the argument about neutrality: the embedding’s gender association for an occupation word tracked the actual share of women in that occupation at Pearson ρ = 0.90 (p < 10−18).

This is not an artefact of one corpus. Lewis and Lupyan reproduced the gender-career effect in 2020 from their own Wikipedia and subtitle corpora, at 1.78 and 1.65. They related the same distributional statistics to 656,636 people’s IAT scores across 25 languages.

Teams use unsupervised methods for exploration, preprocessing, monitoring, and scientific discovery.

Every result depends on a stack of modeling choices that should be made visible.

One study ran that stack in public. On 20 December 2002 Science published a genetic map of human populations by Rosenberg and six co-authors. They genotyped “377 autosomal microsatellite loci in 1056 individuals from 52 populations”. The individuals were then grouped by a program called structure. The paper describes it as placing “individuals into K clusters, where K is chosen in advance but can be varied across independent runs of the algorithm”. K is the analyst’s number, not the data’s. The method it comes from, published in 2000, assumes “K populations (where K may be unknown)”. “Each increase in K split one of the clusters obtained with the previous value.” At K = 5 the clusters lined up with major geographic regions. The sixth was not a region at all. It was “individuals of the isolated Kalash group, who speak an Indo-European language and live in northwest Pakistan”. In one run at K = 6 the algorithm separated the Karitiana instead. Same genotypes, same objective, different sixth cluster.

Use several forms of evidence, because one internal score cannot establish real-world meaning.

The failure being guarded against is not hypothetical, and it has been quantified. Time-series anomaly detectors are scored as standard under a protocol called “point adjustment”. Under it, a uniform random anomaly score reaches F1 of 0.969 on SWaT, 0.965 on WADI, 0.931 on MSL, 0.961 on SMAP and 0.804 on SMD. That beats most published methods on four of the five benchmarks. The identical random scores, left unadjusted, come in at 0.216, 0.109, 0.190, 0.227 and 0.080. Nothing about the detector changed between those two rows; only the rule for counting a hit did. Kim and four colleagues, who ran that experiment in 2022, state it in their abstract without hedging: “even a random anomaly score can easily turn into a state-of-the-art TAD method”. An independent group at Huawei reached the same conclusion that year about range-based metrics generally. Perturbing the data, inspecting exemplars and comparing criteria are not hygiene. They are what stands between you and a headline number of 0.969 produced by noise.

Step four is the one that supplies actual warrant, and the clearest instance of it took a year, a second cohort and a variable nobody had clustered on. In 2000 Perou and colleagues ran unsupervised hierarchical clustering on 65 breast-tumour specimens from 42 individuals across 8,102 genes, in Nature, and obtained subtypes with no outcome label attached to them at all. The external check arrived a year later in PNAS. Sørlie and colleagues clustered 85 microarray experiments covering 78 cancers, 3 fibroadenomas and 4 normal tissues. For the 49 uniformly treated locally-advanced patients, the Kaplan–Meier curves for the subclasses differed in overall survival at P < 0.01, with the basal-like and ERBB2+ subtypes having the shortest survival. Their abstract puts it this way: “Survival analyses on a subcohort of patients with locally advanced breast cancer uniformly treated in a prospective study showed significantly different outcomes for the patients belonging to the various groups, including a poor prognosis for the basal-like subtype and a significant difference in outcome for the two estrogen receptor-positive groups.” Survival was never an input to the clustering. That is precisely why its agreement counts as evidence.

Danger enters when the label changes. An exploratory segment becomes dangerous once it is renamed as a personality type, risk class, or protected-status proxy and then used for consequential decisions.

The first of those renamings has been carried out in a journal. In 2018 Gerlach and three co-authors fitted a Gaussian mixture model to four personality datasets totalling more than 1.5 million participants, and reported “robust evidence for at least four distinct personality types”. Katahira and three colleagues then ran the same procedure on two-dimensional data they had simulated from a single skewed unimodal distribution containing no clusters whatsoever. The fitted mixture returned seven components. Three of them the procedure classified as “meaningful clusters”, covering 45.5% of the samples. Their 2020 commentary states the general result: “even if the target distribution is unimodal and there is no cluster structure, similar results (i.e., emergence of meaningful clusters) can be obtained when the distribution has skewness”. A procedure that returns types from data known to contain none cannot, on its own output, tell you that your data contain them.

Keep the original objective, uncertainty, and validation record attached to the output. If the use changes, the evidence must be reconsidered.