Active Slice Discovery in Large Language Models

AI summary

Overview

Research area: Machine learning reliability and interpretability — specifically slice discovery (finding coherent groups of examples a model gets wrong), active learning, and LLM representation analysis.

Technical level: Intermediate. The paper assumes familiarity with active learning query strategies, classifier training (SVM/MLP), and sparse autoencoder representations, but the central problem is explained in plain terms.

Scope: The paper formalizes "Active Slice Discovery" and empirically tests it on toxicity classification with Llama-3.1-8B, comparing representations, classifiers, and query strategies.

What This Paper Is About

Large language models make systematic errors on particular subsets of data — called error slices — such as performing poorly on toxic comments about a specific demographic. Standard slice discovery methods are fully unsupervised, which makes them hard and imprecise. This paper asks whether a small amount of human annotation, collected actively (one query at a time, chosen by the algorithm), can let a learner identify these error slices accurately while labeling only a tiny fraction of the data.

Key Contributions

1. Formalizes Active Slice Discovery. The paper states the problem setup: given a trained classifier, a small annotated dataset with slice memberships, a larger dataset with only task labels, and a budget of active slice queries, produce a slice membership function. It defines a query strategy that selects which unlabeled example to send to an annotator, and defines slice detection accuracy as the expected indicator that the predicted slice membership matches the true one.

2. Builds a flexible pipeline. The authors implement a modular active slice discovery pipeline in which different base LLMs, representations, classifiers, and active learning strategies can be swapped in combination. Source code is stated to be released upon publication.

3. Empirically evaluates the approach on toxicity classification. Experiments use the Jigsaw toxicity dataset with Llama-3.1-8B, two representations (penultimate-layer embeddings and Llama Scope sparse autoencoder activations on the final layer), two slice classifiers (a feed-forward MLP and a linear SVM), and active learning strategies drawn from the Small-Text library.

4. Identifies best practices. The work compares uncertainty-based, diversity-based, and random query strategies across several slice definitions, reporting which choices are most sample-efficient and which are most stable to train.

Main Findings

• Uncertainty-based query strategies win. Least Confidence, Prediction Entropy, and Breaking Ties consistently achieve higher accuracy with fewer labels than diversity-based strategies (Embedding K-Means, Discriminative Active Learning, Lightweight Coreset) and the Random Sampling baseline, across both raw embeddings and SAE features. The paper notes this aligns with prior work in text classification active learning.

• Dramatic labeling reduction. Active learning can reduce labeling requirements by up to 98% relative to full supervision. The abstract reports that uncertainty-based methods achieve competitive accuracy using 2-10% of the available slice membership information, and the introduction reports reaching comparable accuracy to the full training dataset with as few as 2% of the labels.

• Slice type determines difficulty. Identity-based slices like female and christian can be detected with high accuracy using only a few hundred annotations, whereas reaction-based slices like disagree and sad fail to significantly improve with under 1000 labeled samples. The authors attribute this to slices with similar lexical cues being easier than heterogeneous or sentiment-based slices.

• SAE representations help in specific cases. For the disagree slice, detection rate improves from 0.8 with layer embeddings to 0.83 with SAE representations. SAE features also make active learning training more stable, producing smoother training curves and reducing sensitivity to the choice of uncertainty query strategy.

• Best overall accuracy vs. best simplicity. On the disagree slice, an MLP with active learning on raw layer embeddings reaches 85.8% using 250 labeled examples (out of 12,504). An SVM with Least Confidence queries on raw embeddings reaches 81.0% using 3,500 examples. On SAE features, an MLP with active learning reaches 82.2% using 1,460 examples (out of 12,416), while an SVM with Least Confidence queries reaches 83.0% using 1,000 examples. The MLP achieves the highest observed accuracy but requires careful hyperparameter tuning; the SVM on SAE features is simpler and needs little to no tuning.

Methodology in Plain English

The researchers take a dataset of text (Jigsaw toxicity comments) that already has task labels such as toxic/non-toxic, and a small subset of it that has been annotated for slice membership. They also have a budget of queries they can send to a human annotator, who confirms whether a given example belongs to a particular error slice.

They turn each text example into a numerical representation in two ways: the raw embedding from the penultimate layer of Llama-3.1-8B, and sparse activations from a Llama Scope sparse autoencoder trained on the model's final layer. They then train a one-vs-rest classifier (either a linear SVM or an MLP) to predict slice membership from that representation.

The active learning loop works iteratively: the strategy looks at the currently labeled examples plus the unlabeled pool and picks the single example it believes is most informative — for uncertainty strategies, the one the classifier is least sure about. That example is sent to the oracle, the slice membership comes back, and the classifier is retrained. This repeats until the budget is used up. They test several slices (disagree, likes, female, christian, sad), several strategies, and both representations, measuring test accuracy against the number of labeled examples.

Why This Matters

Impact on research. Slice discovery has largely been treated as an unsupervised problem. This paper opens an active-learning formulation of it, showing that a modest amount of targeted human annotation can substitute for a much larger fully-labeled slice dataset. It also connects LLM mechanistic interpretability — specifically sparse autoencoders — to a practical auditing task.

Real-world applications:

• Content moderation auditing. A platform could check whether a toxicity classifier systematically misjudges comments about a particular demographic, using only a few hundred human confirmations rather than exhaustively labeling slice membership.
• Model evaluation and debugging. Development teams could rapidly surface the subpopulations where a deployed LLM underperforms, guiding where to collect more data or retrain.
• Safety and fairness review. Auditors with limited human-labeling budgets could prioritize which examples to inspect to characterize known failure modes.
• Data collection prioritization. Identifying an error slice early directs annotation effort toward the data that would most improve the model.

Industry relevance. The reported 98% reduction in labeling relative to full supervision is directly relevant to teams that have large unlabeled logs but scarce annotator time. The finding that simple models (SVM) on SAE features are competitive also lowers the engineering cost of adopting the approach.

Future Directions

• Extending beyond toxicity classification. The paper only reports experiments on the Jigsaw toxicity dataset with Llama-3.1-8B; whether the same strategy rankings hold for other tasks, models, and modalities is not established.

• Handling hard, sentiment-based slices. The disagree and sad slices failed to improve significantly with under 1000 labeled samples, raising the question of what query strategies or representations would be needed for heterogeneous, non-lexically-cued slices.

• Multi-slice and multi-label querying. The formal setup allows for k slices, but the paper's detailed query strategy comparisons fix attention on the disagree slice. How to allocate a single annotation budget across several interacting slices remains open.

• Better representations for slice discovery. SAEs improved the disagree slice's detection rate from 0.8 to 0.83 and stabilized training, suggesting further gains may come from representation design rather than query strategy design.

Target Audience

Researchers and practitioners working on model reliability, fairness auditing, and interpretability — particularly those who already understand active learning or slice discovery and want to know whether eliciting slice membership labels is a practical alternative to unsupervised grouping. It is also useful for ML engineers with large unlabeled data and limited annotation budgets who want a concrete sense of how few labels are needed and which query strategies to try.

Authors’ abstract

Large Language Models (LLMs) often exhibit systematic errors on specific subsets of data, known as error slices. For instance, a slice can correspond to a certain demographic, where a model does poorly in identifying toxic comments regarding that demographic. Identifying error slices is crucial to understanding and improving models, but it is also challenging. An appealing approach to reduce the amount of manual annotation required is to actively group errors that are likely to belong to the same slice, while using limited access to an annotator to verify whether the chosen samples share the same pattern of model mistake. In this paper, we formalize this approach as Active Slice Discovery and explore it empirically on a problem of discovering human-defined slices in toxicity classification. We examine the efficacy of active slice discovery under different choices of feature representations and active learning algorithms. On several slices, we find that uncertainty-based active learning algorithms are most effective, achieving competitive accuracy using 2-10% of the available slice membership information, while significantly outperforming baselines.