Rethinking SAE Evaluation: An Atomic Interpretable Unit Framework Reveals Hidden Polysemanticity and Redundancy

AI summary

Overview

Research area: Mechanistic interpretability of large language models, specifically how to evaluate sparse autoencoders (SAEs) used to decompose model activations into interpretable features.

Technical level: Advanced. The paper combines a formal framework (mapping conditions between concepts and units), new evaluation metrics, and a large empirical benchmark; the main text is readable with a background in interpretability, but the core arguments rely on the appendices.

Scope (one sentence): The paper argues that SAE evaluation lacks a stated global objective, proposes "atomic interpretable unit (AIU) basis" as that objective with five derived criteria, and uses new dominance-aware metrics to show that the most computationally important SAE features are systematically less monosemantic and more redundant than the rest.

What This Paper Is About

SAEs were introduced to recover monosemantic features from superposed model activations, but as they became tools for attribution, circuit discovery, and intervention, researchers added many other metrics—reconstruction fidelity, sparsity, dead feature rates, disentanglement—without ever formalizing what the feature set as a whole is supposed to be. The authors argue monosemanticity is only a local property of single features and therefore cannot justify these global metrics, so they define the missing objective: SAE features should form an atomic interpretable unit basis, a coordinate system in which each semantically meaningful concept maps to a distinct, usable unit. They then build metrics that test this objective and show that existing evaluations miss important failures in the features that matter most for model computation.

Key Contributions

1. A formal objective for SAE evaluation. The paper defines the atomic interpretable unit (AIU) basis and derives five necessary criteria from it: non-redundancy (C1), completeness (C2), monosemanticity (C3), no dead units (C4), and per-input sparsity (C5). Criteria C1–C4 come from injectivity/surjectivity of the mappings between concepts and units; C5 follows from the separate requirement that explanations be cognitively accessible.

2. New dominance-aware metrics. It introduces DAME (Dominance-Aware Monosemanticity Evaluation), which stratifies features by mean dominance and scores explanations by "perfect ranking accuracy" over low/medium/high activation triples, plus decoder-based redundancy metrics with a theoretical and geometric argument for why non-redundancy should be enforced on decoder directions rather than encoder weights.

3. A large open-source benchmark. The authors evaluate 600+ SAEs across 7 SAE families, sampling 28,000+ features, covering existing families such as SAEBench, JBloom, and CanRager along with a suite of newly trained SAE variants spanning multiple architectures and three different LLMs. Code and checkpoints are released on GitHub and HuggingFace.

4. Identification of two previously undetected failure modes. "Functional polysemanticity" driven by dominance, and "feature sharding," in which features share nearly identical decoder directions but fire on complementary inputs—allowing an SAE to look healthy on common diagnostics while its effective semantic dictionary collapses to far fewer directions.

Main Findings

• Dominant features are more polysemantic. Across all evaluated SAEs, top-quartile (Q1) features by mean dominance score lower on DAME than Q2–Q4: 61.8% versus 64.1% perfect ranking accuracy, significant at the per-SAE level (Wilcoxon p = 6.76 × 10⁻⁸, Cohen's d = −0.278), with 65.2% of SAEs individually showing a Q1 deficit.
• The deficit is broad but not universal across architectures. It is statistically significant in 5 of 7 architectures (BatchTopK, TopK, Matryoshka, PAnneal, Standard ReLU) and not significant for Gated and JumpReLU, which the authors suggest may reflect the high L₀ produced by the default sparsity hyperparameters used for those architectures. Per-architecture Q1→Q4 accuracy, for example, goes from 61.5 to 69.9 for BatchTopK and from 59.2 to 64.6 for TopK.
• Dominant features are more redundant. At a decoder cosine threshold of 0.7, median Q1 endpoint enrichment is 1.64× (95% SAE-bootstrap CI 1.51–1.74), with 71.5% of SAEs above the 1× null. For Standard ReLU SAEs, 26.1% of top-10 features participate in Q1 redundant pairs (4.7 pairs within the top-10), and 20.5% of top-100 features do (296.3 pairs within the top-100).
• Redundant dominant features differ at the encoder. Q1 features have the lowest mean encoder similarity (0.056 versus 0.109–0.135 for Q2–Q4), and even among redundant pairs Q1 pairs have encoder similarity 0.428 versus 0.661 for Q2–Q4—evidence that decoder-aligned features can be gated very differently.
• Activation is highly concentrated. Even in SAEs with 4,000–128,000 features, the 10 most frequently firing features contribute a median 7.8% of per-input activation, with some SAEs above 30%; the top-30 median is 13.0%.
• Sparsity correlates with dominance concentration. Higher sparsity (lower L₀) correlates with higher dominance across all evaluated SAE families; in GPT2-Small SAEs, 17 of 18 width-architecture combinations (94%) show positive correlations.
• DAME discriminates better than prior scoring. In a controlled test of high-versus-middle activation reversals that preserve binary firing support, DAME ranks the correct explanation above the reversed one for all 80 synthetic features, versus 67 for discretized magnitude scoring and at most 22 for binary/fuzz-style scoring.
• A functional sharding case. In a TopK-65k SAE on Pythia-160M, feature 41902 (definite articles) and feature 46571 (possessive/genitive determiners) have encoder and decoder cosine similarities of 0.2066 and 0.7405, with 316 versus 57 firings and only 7 co-firings (Jaccard 0.019). Injecting their decoder directions at positions where both gates were inactive raised the target token logit by +0.556 and +0.611 on average (15/18 and 16/18 positive prompt-level effects), with an effect-vector cosine of 0.943.
• Sharding is distinct from absorption. Across decoder-aligned pairs, 21.5% fire at largely separate positions (sharding-like) and 15.3% have one feature's firing positions largely contained within the other's (absorption-compatible); the two can coexist.
• Judge-free validation agrees. Across 210 synthetic SAEs at widths 128–1024, Q1 features show lower prevalence-corrected held-out support containment (one-sided SAE-level sign test, p = 1.6 × 10⁻¹²), and a correlation-gap analysis finds a significant Q1 deficit in 71 of 105 SAEs.

Methodology in Plain English

The authors start by asking what an explanation needs in order to be meaningful: distinct concepts must map to distinct units, otherwise you cannot tell which concept a unit is responding to. Pixel patches roughly satisfy this in vision, but tokens and neurons do not in language models—tokens carry many unrelated roles and neurons are superposed. SAEs are attractive because their feature set is learned rather than fixed by architecture, so the paper turns that intuition into five checkable criteria.

For measurement, three criteria map onto familiar metrics: reconstruction fidelity for completeness, firing statistics for dead units, and mean L₀ for sparsity. Monosemanticity and non-redundancy are harder because they are properties of individual features or feature pairs, so any dictionary-level score needs an aggregation rule. The authors argue the usual uniform weighting is wrong because feature usage is extremely imbalanced, and instead weight by dominance—each feature's share of total activation on a given input.

DAME therefore samples features across dominance quartiles and, for each feature, builds validation questions from three inputs with low, medium, and high activation. An LLM is shown an explanation and must rank the three inputs by expected activation without seeing the values; the score is how often the ordering is exactly right. This avoids the failure where a rare feature gets an all-negative validation set and any explanation looks perfect. Redundancy is measured with cosine similarity between decoder directions, justified by the argument that sparsity already suppresses encoder-side duplication while decoder duplication is not penalized by the standard objective. The authors also check that decoder similarity tracks semantic similarity, finding a pooled Spearman correlation of 0.42 (p < 10⁻¹⁰) with explanation-embedding similarity.

Why This Matters

The paper reframes SAE evaluation from a checklist of loosely connected metrics into a stated objective with derived criteria, and it shows that a widely used diagnostic picture ("this SAE has high sparsity and few dead features") can coexist with a semantically collapsed dictionary. Because the problematic features are exactly the ones that dominate activation and therefore dominate attribution, circuit tracing, and interventions, the result directly threatens the downstream uses that motivated SAEs in the first place. The authors recommend dominance-stratified evaluation—monosemanticity across dominance quantiles and decoder similarity among high-dominance features—as a standard check before SAE features are deployed.

Real-world applications implied by the work:

• Auditing interpretability tools before deployment in model debugging, red-teaming, and safety review pipelines.
• Circuit tracing and attribution studies that rely on the features contributing most to computation being trustworthy.
• Activation steering and intervention for model behavior control, where duplicated or polysemantic dominant features could misattribute causal effects.
• Model documentation and compliance reporting, where claims about identified concepts need a defensible evaluation standard.

Industry relevance: teams that publish or consume SAE feature sets (model developers, interpretability tooling vendors, and safety-focused labs) would need to add dominance-stratified checks to their release criteria, and SAE training objectives or architectures would need to be redesigned to penalize decoder redundancy and polysemanticity in high-dominance features.

Future Directions

• Training objectives and architectures that target high-dominance features directly, since the observed failure modes let SAEs satisfy sparsity and dead-feature penalties without improving interpretability.
• Whether sparsity constraints genuinely decompose features or mainly concentrate computation into fewer, increasingly dominant ones—an open question the authors explicitly raise.
• Extending the dominance-aware analysis beyond the models evaluated, given the stated limitations around model scale, the proxy nature of the LLM judge, conditional ground-truth concept sets, and hyperparameter transfer.
• Separating sharding from absorption empirically and determining which training incentives produce each, since the two patterns can coexist among decoder-aligned pairs.

Target Audience

SAE researchers and interpretability engineers who train, benchmark, or deploy dictionary-learning features; mechanistic interpretability practitioners using features for attribution, circuit discovery, or intervention; and evaluation-focused ML researchers interested in how a formal objective can replace ad hoc metric collections. Readers without background in superposition and auto-interpretability pipelines will find the appendices necessary, and readers looking for a short empirical result should focus on Sections 3.1, 4.1, and 4.2.

Authors’ abstract

Sparse autoencoders (SAEs) were originally motivated by discovering monosemantic features, a property that characterizes individual features but does not specify what the learned feature set should look like as a whole. As SAEs have increasingly been adopted for attribution, circuit discovery, and intervention, researchers have begun evaluating metrics that describe global properties of the feature set, such as sparsity, dead feature rates, reconstruction fidelity, and disentanglement, a diverse set that extends far beyond monosemanticity. However, this implicit shift beyond monosemanticity has not been formalized as a clear, well-defined objective that comes before these evaluation metrics. Without such an objective, existing metrics remain ad hoc, with their necessity unjustified. In this work, we formalize this ultimate objective by introducing a new paradigm for SAE evaluation: requiring that SAE features, and more generally dictionary-based decompositions of model representations, form an atomic interpretable unit basis, a coordinate system for semantically meaningful explanation. Under this paradigm, we organize existing and new measurements such as DAME (Dominance-Aware Monosemanticity Evaluation) as proxy metrics and introduce a comprehensive benchmarking framework. Evaluating hundreds of SAEs across multiple architectures and model families, we uncover failure modes in which some SAEs appear monosemantic under existing evaluation metrics while remaining functionally polysemantic: the most dominant features, those contributing most to model computation, are systematically less interpretable and more redundant. These failure modes warrant attention from SAE researchers as targets for improved training and architectures, and motivate dominance-aware evaluation as a standard check for practitioners before SAE features are deployed for attribution, circuit tracing, or intervention.