ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts

AI summary

Overview

Research area: Computer vision, with overlap into model interpretability and dataset auditing.

Technical level: Intermediate. Readers will get the most from this paper with some familiarity with vision-language models (CLIP), sparse autoencoders, and the standard framing of spurious correlations and dataset bias in image classification.

One-sentence scope: The paper introduces ConceptScope, a two-stage automated pipeline that trains Sparse Autoencoders on CLIP-ViT features to build an interpretable visual concept dictionary, then labels each concept per class as target, context, or bias in order to audit datasets and diagnose model robustness.

What This Paper Is About

Machine learning image datasets are skewed: certain visual patterns co-occur with certain class labels far more often than they should, and models learn to exploit those shortcuts instead of the actual object. Finding these biases normally requires expensive, fine-grained human attribute annotations that do not scale to large datasets.

ConceptScope addresses this by automatically discovering human-interpretable visual concepts from a vision foundation model and quantifying how each concept relates to each class label, so that biases can be identified, localized, and used to evaluate model robustness without any manual annotation step.

Key Contributions

1. A concept-categorization framework. ConceptScope systematically extracts visual concepts, categorizes them by relevance to target classes (target, context, bias), and quantifies their distributions across a dataset.

2. Empirical validation of SAE activations as concept extractors. The authors show through binary attribute classification and segmentation experiments that SAE latent activations are reliable, interpretable proxies for the presence of visual concepts.

3. Bias detection and robustness diagnosis in practice. They demonstrate that the framework recovers known annotated biases and uncovers previously unannotated ones, and that grouping test samples by concept strength produces subgroups with distinct accuracy profiles.

4. A public implementation. Code is released at https://github.com/jjho-choi/ConceptScope.

Main Findings

• SAE beats caption-based concept prediction. On six annotated datasets (Caltech101 for objects, DTD for textures, Waterbird for backgrounds, CelebA for facial attributes, RAF-DB for emotions, Stanford40 for actions), the SAE-based method reached an average F1 of 0.72 and AUPRC of 0.76, versus average F1 of 0.43 for BLIP-2 and 0.58 for LLaVA-NeXT. Per-dataset SAE F1 was 0.83 (Caltech101), 0.57 (DTD), 0.78 (Waterbird), 0.81 (CelebA), 0.55 (RAF-DB), and 0.78 (Stanford40). Results were stable across four random seeds with a standard deviation of 0.01.

• Activation strength tracks concept presence. Treating CLIP image-text similarity as pseudo-ground truth, SAE image-level activations correlated with it at Pearson r = 0.71 and Spearman rho = 0.65 across the six attribute-labeled datasets.

• Spatial attributions give usable segmentation masks. On 2,000 ADE20K validation images spanning 150 semantic categories, SAE-derived masks achieved AUPRC of 0.399, versus 0.302 for LLaVA-NeXT and 0.098 for BLIP-2. Results were averaged over four SAEs trained with different seeds (standard deviation 0.002). The authors note performance drops on small objects because the patch resolution is 16x16, and that the SAE produces masks for all concepts in a single forward pass.

• Known biases are detected at state of the art. On the bias discovery task with Precision@10, ConceptScope scored 100.0% on Waterbirds, 100.0% on CelebA, and 72.9%, 73.1%, and 74.0% on NICO++ at bias severity levels 75, 90, and 95 respectively. ViG-Bias scored 100.0%, 100.0%, 60.0%, 66.7%, 65.0%; FACTS scored 100.0%, 100.0%, 55.0%, 60.8%, 61.0%; DOMINO scored 90.0%, 87.0%, and 24.0% at all three NICO++ levels. ConceptScope results were averaged over four independently trained SAEs with standard deviation below 0.02.

• New biases surface in unannotated datasets. In ImageNet-1K, SUN397, and Food101 the method found object associations ("necklaces" with "mannequins"), cultural correlations ("bridegroom" with "East Asian cultural" contexts), and location-specific patterns ("ice skating rink" with "New York City"). The average number of bias concepts per class was 2.45.

• Robustness can be assessed without an external OOD set. Splitting ImageNet validation and ImageNet-Sketch by high/low target and bias concept strength produces four subgroups. Across 34 pretrained vision models, Group 1 (high target, high bias) and Group 2 (high target, low bias) dominate both datasets, and accuracy descends consistently from Group 1 to Group 4. ConvNeXt-Large achieved the highest average accuracy and performed best on Group 4.

• Concepts transfer beyond the training dataset. Although the SAE was trained on ImageNet-1K without class labels, its concepts generalized to datasets used in CLIP zero-shot evaluation, including Food101 and SUN397. Applied to the MSCOCO 2017 test set in a multi-label setting, it found that "cat" associates with indoor environments, "dog" with couches and living rooms, and "bird" with bird feeders and trees.

Methodology in Plain English

The pipeline has two halves.

Building the concept dictionary. The authors take a strong pretrained vision encoder (CLIP-ViT-L with patch size 14) and extract its intermediate patch-level token embeddings. They then train a Sparse Autoencoder on those embeddings using ImageNet-1K. An SAE has a linear encoder, a nonlinear activation (ReLU), and a decoder; its latent dimension is deliberately much larger than the input embedding, using an expansion factor of 16 or 32. Training minimizes a reconstruction loss plus an L1 penalty controlled by a sparsity weight, which pushes each latent unit to respond to one narrow, monosemantic pattern rather than a mixture. Each row of the decoder corresponds to one learned concept, and the encoder output indicates how strongly that concept fires on a given patch.

Naming the concepts. To make each latent human-readable, the authors collect its top-activating images, pull segmentation masks following the procedure of Lim et al., and pass those visual inputs to GPT-4o to generate short descriptions. This annotation step is fully automated apart from light prompt tuning and is used only for human interpretability, not in the categorization itself.

Sorting concepts by role. For each class, an alignment score combines two measurements: necessity (how much prediction confidence drops when the concept's spatial region is masked out) and sufficiency (how much confidence survives when only that region is kept). Concepts whose combined score exceeds the class mean plus a threshold factor times the class standard deviation are labeled target; everything else is context. Confidence here is the cosine similarity between the CLIP text embedding of the class label and the image embedding.

Separating bias from ordinary context. The authors then measure concept strength: the average SAE encoder activation for a concept over all images of a class. Among non-target concepts, any concept whose strength exceeds one standard deviation above the mean is labeled a bias concept.

Why This Matters

Dataset bias is usually diagnosed indirectly, by training a classifier and inspecting where it fails, which cannot reveal which training samples carry the biasing feature or how prevalent it is. ConceptScope inspects the training data directly, produces a class-level breakdown of which concepts are essential versus incidental, and localizes the offending image regions. Because the SAE can be reused across datasets without retraining, the approach offers a cheap auditing primitive for the dataset curation problem that scaling laws have made increasingly prominent.

Real-world applications:

• Dataset auditing before training. Teams can profile a dataset's class-wise concept distributions and flag over-represented backgrounds or co-occurring objects before committing compute to a training run.
• Model diagnostics and robustness reporting. The four concept-strength subgroups provide an accuracy breakdown on the original test set, useful in domains like food or scene classification where no curated out-of-distribution benchmark exists.
• Fairness and bias review. The method surfaces cultural and geographic correlations, such as "bridegroom" with East Asian cultural contexts, that are hard to catch with predefined gender or demographic categories.
• Domain-specific analysis. The paper points to CytoSAE and Mammo-SAE as evidence that the same SAE approach applies to medical imaging when the SAE is retrained on the target domain or paired with a more suitable backbone.

Industry relevance: The framework targets the practical bottleneck of dataset curation, which the paper notes is currently done by training models on multiple curated dataset versions under a fixed protocol and comparing performance, a resource-intensive and indirect process. An automated concept-level audit that runs once per dataset and transfers to others is directly useful to teams building or procuring large image corpora.

Future Directions

• Reducing dependence on the CLIP representation. The authors state that discovered concepts are bounded by what CLIP encodes, which shows up as weaker concept prediction on domain-specific datasets such as RAF-DB (emotion) and DTD (texture). Fine-tuning the foundation model on the target domain is the suggested remedy.

• Improving localization granularity. The current segmentation masks are patch-level and coarse, with a 16x16 resolution that hurts small-object performance. Better localization would sharpen the necessity and sufficiency scores that drive target/context separation.

• Extending to settings beyond single-label classification. The paper demonstrates a multi-label adaptation on MSCOCO 2017 by extracting object mentions from ground-truth captions, leaving open how the framework should handle richer or non-object supervision.

• Domain adaptation for specialized data. For data far from ImageNet, such as medical images, the open question is how best to retrain the SAE and select a backbone, given that prior work like CytoSAE and Mammo-SAE has shown feasibility but the paper does not specify a general recipe.

Target Audience

Researchers and practitioners working on dataset curation, dataset bias and fairness, and trustworthy computer vision will get the most from this paper. It is also relevant to interpretability researchers interested in Sparse Autoencoders applied outside language models, and to ML engineers who need a concrete, code-backed tool for auditing image datasets and producing robustness breakdowns without an external out-of-distribution test set.

Authors’ abstract

Dataset bias, where data points are skewed to certain concepts, is ubiquitous in machine learning datasets. Yet, systematically identifying these biases is challenging without costly, fine-grained attribute annotations. We present ConceptScope, a scalable and automated framework for analyzing visual datasets by discovering and quantifying human-interpretable concepts using Sparse Autoencoders trained on representations from vision foundation models. ConceptScope categorizes concepts into target, context, and bias types based on their semantic relevance and statistical correlation to class labels, enabling class-level dataset characterization, bias identification, and robustness evaluation through concept-based subgrouping. We validate that ConceptScope captures a wide range of visual concepts, including objects, textures, backgrounds, facial attributes, emotions, and actions, through comparisons with annotated datasets. Furthermore, we show that concept activations produce spatial attributions that align with semantically meaningful image regions. ConceptScope reliably detects known biases (e.g., background bias in Waterbirds) and uncovers previously unannotated ones (e.g, co-occurring objects in ImageNet), offering a practical tool for dataset auditing and model diagnostics.