Sparse Autoencoders are Topic Models

AI summary

Overview

Research area: Interpretability of foundation-model embeddings; topic modeling; computer vision and NLP dataset analysis.

Technical level: Advanced (the central claim is a probabilistic derivation showing the sparse autoencoder objective is a maximum a posteriori estimator of a proposed generative topic model).

One-sentence scope: The paper reframes sparse autoencoders (SAEs) as topic models by introducing a continuous topic model inspired by LDA, deriving the SAE objective from it, and validating the view with a framework that produces coherent topics on text and image data.

What This Paper Is About

Sparse autoencoders are widely used to break down model embeddings into human-readable features, but their role is contested: they have been criticized for failures in steering and for underperforming linear probes. The authors ask how SAEs should be understood and how their strengths can best be exploited. Their answer is that SAEs are naturally topic models — systems that represent each datapoint as a mixture of themes recurring across a dataset — rather than sets of individually steerable directions.

Key Contributions

1. A theoretical bridge between topic models and SAEs. The authors introduce a continuous topic model (CTM) inspired by Latent Dirichlet Allocation (LDA) that operates on document embeddings instead of words, and they derive the SAE objective as a MAP estimator under this model. Fixed-sparsity SAEs such as TopK (Gao et al., 2025) and BatchTopK (Bussmann et al., 2024) are shown to admit an analogous derivation that replaces the high-activity limit with a hard support constraint.

2. SAE-TM, a topic-modeling framework built on the theory. It (1) trains an SAE to learn reusable "topic atoms," (2) interprets those atoms as word distributions on downstream data via a learned word-emission matrix, and (3) merges atoms into any number of topics without retraining.

3. Empirical comparison against strong baselines. SAE-TM is evaluated on five text datasets and three image datasets against eight neural topic models, reporting topic coherence and diversity.

4. Large-scale thematic analysis of image datasets and of Japanese woodblock prints. The authors compare the thematic composition of four widely used image datasets and, per the contributions, trace topic changes over time in Japanese woodblock prints (the detailed results for the latter fall in the portion of the paper not included in the supplied text).

Main Findings

• Coherence on text: SAE-TM achieves the highest coherence scores on the five text datasets. At 50 topics it reaches C_I 54.31 and C_R 77.25; at 100 topics C_I 51.48 and C_R 78.01; at 200 topics C_I 46.63 and C_R 75.71; at 300 topics C_I 43.50 and C_R 74.22; at 500 topics C_I 40.49 and C_R 71.22. The next-best coherence performers are typically AVITM (50 topics: C_I 38.72, C_R 69.05) and CombinedTM (50 topics: C_I 40.90, C_R 70.24).

• Coherence stability as topic count grows: TSCTM starts with the highest diversity (D 3.87 at 50 topics) and a C_R of 69.75, but its coherence falls to C_R 17.67 at 500 topics. SAE-TM maintains high and stable coherence even at 500 topics.

• Diversity on text: SAE-TM consistently ranks second on topic diversity, trailing only TSCTM. Its diversity score is 3.67 at 50 topics, 3.64 at 100, 3.60 at 200, 3.59 at 300, and 3.57 at 500.

• Coherence on images: On three image datasets, SAE-TM again produces significantly more coherent topics, reaching C_R 85.05 at 50 topics, 85.67 at 100, 85.53 at 200, 85.05 at 300, and 84.43 at 500, compared with TSCTM's 80.40 at 50 topics and 39.81 at 500.

• Diversity on images is weaker: SAE-TM's image diversity (D 3.70 at 50 topics down to 3.53 at 500) is lower than some baselines, such as DecTM (D 3.94 at 50) and TSCTM (D 3.91 at 50). The authors attribute this to modality effects — image embeddings focus on a few foreground objects, and some SAE features capture concepts that language explains poorly, binding high-frequency words during interpretation.

• Embedding-based baselines also work: Among the baselines, only CombinedTM and FASTopic operate on embeddings like SAEs. CombinedTM performs well at low topic counts, while FASTopic remains stable across topic counts, supporting the potential of learning topic models from image embeddings alone.

• Thematic structure of image datasets: Analyzing ImageNet, CC3M, CC12M, and YFCC-15M, the top-10 topics by variance across datasets show ImageNet with more plants ("Delicate Plants"), animals ("Fluffy Animals," "Wildlife"), and technical tools ("Containers and Packaging"), but significantly fewer images of humans ("Human Interaction"). CC3M and CC12M contain more text and typographic elements, a trend particularly pronounced for CC12M. YFCC features many urban scenes ("Urban Environment") and, with CC3M, many musical performances ("Live Performance"); like ImageNet, YFCC features many natural landscapes ("Lush Landscape").

• Sources of dataset difference: The authors conclude that differences arise partly from dataset construction and partly from image sources, since even CC3M and CC12M, built with similar methodology, diverge.

• Efficiency: Training an SAE on 50M embeddings (the Twitter dataset) takes 10 minutes on a single GPU, and interpretation takes 15 minutes.

Methodology in Plain English

The authors start by writing down a generative story for how a document embedding could be produced, deliberately mirroring how LDA generates a bag of words. In their continuous topic model, each document draws a mixture over topics from a Dirichlet distribution; the number of contributions is Poisson-distributed; each contribution picks a topic, a direction sampled from a Gaussian around that topic's mean, and a strength sampled from a Gamma distribution. Contributions are summed, with Gaussian noise added, to produce the observed embedding. The expected embedding is linear in the topic mixture, exactly paralleling LDA's βᵀθ structure.

They then show that if many tiny contributions accumulate (the high-activity, small-contribution limit), if topic directions are concentrated, and if per-topic strengths are independent, the negative log-posterior collapses into the standard SAE loss: reconstruction error plus an L1 penalty. That means the SAE is the topic model, with each feature acting as a topic atom and its activation as the inferred topic weight.

Because SAEs have far more features (≫1,000) than typical topic models have topics, and because SAE features are not word distributions, the authors add two post-hoc interpretation layers on top of a frozen SAE. First, they learn a word-emission matrix that maps each feature to a distribution over vocabulary items, using a bag-of-words likelihood with a background unigram prior (π = 0.3 in all experiments) and inverse-document-frequency weighting. Second, they merge features into a smaller number of topics by clustering topic embeddings — built as weighted sums of word embeddings such as word2vec or GloVe, or alternatively from the SAE decoder weights — with k-means, and aggregating the word distributions. For denoising, they keep only the top-p vocabulary (p = 0.9).

Evaluation uses two coherence metrics — intruder detection accuracy (C_I) and overall topic rating (C_R) — judged by the LLM Phi-4, plus a diversity metric based on average word mover distance between topics' top-20 words. Text datasets are preprocessed with NLTK (lemmatization, stopword filtering, top 5,000 lemmas) and embedded with Granite-R2. Images are embedded with ViT-B-16-SigLIP from OpenCLIP and captioned with InternVL3.5-14B. SAEs are trained with expansion factor 4 (dictionary size 3072), L1 penalty 2, batch size 1000, 50,000 steps, and learning rate 0.001; interpretation uses vocabulary size 5000, batch size 1000, learning rate 0.01, and 50 to 200 epochs.

Why This Matters

Impact on research. The paper offers a probabilistic explanation for why SAEs capture thematic structure but disappoint at single-feature steering: features are thematic components whose activations combine to explain an embedding, not monosemantic controllable directions. This reframes what SAEs are good for — discovering and organizing unknown themes — and supplies a reusable, retraining-free path from a pretrained SAE to topic models at any granularity.

Real-world applications:

• Dataset auditing and selection. Comparing ImageNet, CC3M, CC12M, and YFCC-15M reveals systematic contrasts (object-centric versus human-centric emphasis, text and typographic content, urban versus landscape scenes), which the authors connect to dataset rebalancing and data selection.
• Avoiding expensive attribute labeling. SAE topic models sidestep the need for attribute annotation via multimodal LLMs or specialized models.
• Large-scale multimodal thematic analysis. The framework applies to text and images alike, since it operates in embedding space rather than on raw tokens.
• Cultural and historical analysis. The authors apply their topic models to detect changing themes in Japanese woodblock prints across periods.

Industry relevance. Practitioners who already train SAEs on internal model activations can reuse them as topic models — merging atoms into whatever number of topics a stakeholder needs without retraining — making large-scale inspection of proprietary text and image corpora cheaper than attribute-labeling pipelines.

Future Directions

• Improving SAE feature interpretation for images, so topics bind fewer high-frequency, non-thematic words; the authors explicitly flag the weaker image diversity as warranting this.
• Tighter integration with existing topic-modeling tooling, extending work such as Zheng et al. (2025), which uses SAE features as input tokens, given that this paper establishes the SAE objective itself is a MAP estimator of a topic model.
• Scaling the analysis to finer granularity, from 100 merged topics down to atomic SAE features, and expanding beyond the four datasets studied.
• Tracing downstream effects of dataset composition, which the authors say motivates applications in dataset rebalancing and data selection.

Target Audience

Researchers and practitioners in mechanistic interpretability, topic modeling, and dataset curation — particularly those working with foundation-model embeddings, multimodal LLM pipelines, or large-scale text and vision corpora. Readers need familiarity with probabilistic modeling, latent variable models, and autoencoders to follow the derivation in Section 3; the empirical sections are accessible to a broader audience interested in dataset analysis.

Authors’ abstract

Sparse autoencoders (SAEs) are used to analyze embeddings, but their role and practical value are debated. We propose a new perspective on SAEs by demonstrating that they can be naturally understood as topic models. We propose a continuous topic model (CTM) inspired by Latent Dirichlet Allocation (LDA) for embedding spaces and derive the SAE objective as a maximum a posteriori estimator under this model. This view implies SAE features are thematic components rather than steerable directions. To confirm our theoretical findings, we introduce SAE-TM, a topic modeling framework that: (1) trains an SAE to learn reusable topic atoms, (2) interprets them as word distributions on downstream data, and (3) merges them into any number of topics without retraining. SAE-TM yields more coherent topics than strong baselines on text and image datasets while maintaining diversity. Finally, we analyze thematic structure in image datasets and trace topic changes over time in Japanese woodblock prints. Our work positions SAEs as effective tools for large-scale thematic analysis across modalities. Code is available at https://github.com/ExplainableML/SAE-TM .