User Embeddings are Superpositions of Interpretable Behavioral Modes

AI summary

Overview

• Research area: Machine learning interpretability, specifically mechanistic interpretability applied to user embeddings used in recommendation and personalization systems.
• Technical level: Intermediate. The paper combines sparse dictionary learning (SAEs), generative modeling of behavior, and empirical retrieval evaluation; readers should be comfortable with embeddings, linear algebra, and basic recommender-system terminology.
• Scope: The paper argues that dense user embeddings are sparse superpositions of interpretable "behavioral modes," shows how to recover those modes with sparse autoencoders post-hoc, validates recovery against known ground truth on synthetic data, and evaluates natural-language audience retrieval on two public datasets.

What This Paper Is About

Recommendation and personalization systems represent each user as a dense vector, but these vectors are opaque: it is hard to say why a recommendation was made, what a vector contains, or to search the user space with natural language. The authors argue that a user embedding is actually a sparse mixture of a small number of interpretable behavioral modes (for example, "dog lover" or "new parent"), and that this structure follows from a simple generative model of user behavior. They then recover those modes with sparse autoencoders, automatically name them with an LLM, and use the names to run zero-shot natural-language audience queries over embeddings that were never trained on text.

Key Contributions

1. A generative justification for superposition in user embeddings. The authors show that if each interaction is drawn from a per-user mixture of behavioral-mode distributions, then any embedding formed by sample-mean pooling (or linearly transformed mean pooling) is, in expectation, a sparse linear combination of mode centroids. The argument extends approximately to transformer encoders via the superposition hypothesis, with the caveat that single-position readouts (last-token, [CLS]) are not covered directly.

2. Ground-truth validation of mode recovery on synthetic data. Using 75 predefined behavioral modes and 100,000 synthetic users with known mixture weights, the paper measures recovery directly and compares SAEs against random projections, sparsified PCA and FastICA, K-Means encoding, NMF, and the soft-frozen-decoder SAE baseline from Korznikov et al. (2026), on both mean-pooled product embeddings and SASRec-encoded embeddings.

3. A post-hoc, model-agnostic interpretation pipeline. The method decomposes pre-trained embeddings with an SAE, auto-interprets each latent from its top-activating users with an LLM, and scores latents against natural-language queries at retrieval time, all without modifying the upstream embedding system or requiring text-aligned training.

4. Zero-shot natural-language audience retrieval and audience characterization on real data. The pipeline is evaluated on Open e-commerce 1.0 (1.8M purchases from 5027 U.S. consumers) and MovieLens 1M (1M ratings, 6040 users, 3706 movies), including a comparison against a text-embedding baseline and an analysis of feature redundancy in top-k audience descriptions.

Main Findings

• SAEs lead in the high-superposition regime on synthetic data. When the embedding dimension m is below the mode count K = 75, the SAE matches or exceeds every classical baseline on mode recovery and stays close to the oracle ceiling on code–ground-truth alignment. PCA recovers almost no modes at any m (≤ 9%), as expected given its orthogonal components.
• K-Means encoding is competitive but capacity does not close the gap. K-Means encoding holds steady near 80% recovery (tied with the SAE at m = 16) and matches or slightly exceeds the SAE on alignment at m ≤ 24. A 512-component K-Means recovers more directions than the SAE at m ≤ 48 but underperforms the 75-component reference on code–ground-truth alignment at every m ≥ 32 (paired Wilcoxon p ≈ 0.03 at m ≥ 48).
• Feature splitting is measurable and fixable. Raw SAE median per-feature correlation is ≈ 0.35 and rises to ≈ 0.70 at m = 128 after merging features whose decoder directions have cosine similarity > 0.45, confirming the splitting is artifactual.
• Merged SAE latents are more monosemantic. At m = 128 the merged SAE reaches 0.53 mean monosemanticity versus 0.39 for K-Means encoding and 0.28 for raw embedding dimensions. FastICA reaches 0.75 because the generative model's independent-mode assumption matches its objective, but it recovers fewer decoder directions at high cosine thresholds.
• Superposition analysis transfers to transformer embeddings. On SASRec embeddings at d = 256 trained on the synthetic sequences, the oracle ceiling is 0.78 versus 0.85 for mean-pooled product embeddings. The SAE leads on code–ground-truth alignment (0.58, versus 0.53 for FastICA, 0.45 for K-Means encoding, 0.37 for LDA on raw counts, and 0.25 for PCA) and recovers 73 of 75 modes at cosine ≥ 0.7, while FastICA and K-Means encoding drop from 95% to 59% and from 79% to 52%.
• On Open e-commerce, the SAE wins on every retrieval-quality metric. For the 6 category tasks the SAE records FVU 0.075, Lift 1.93×, AUROC 0.705, AUROC_scored 0.678, and Coverage 0.735, versus FastICA (0.083, 1.86×, 0.695, 0.660, 0.770) and K-Means encoding (0.388, 1.85×, 0.680, 0.663, 0.614). For the 2 life-event tasks the SAE records Lift 5.30× and AUROC 0.715 versus FastICA at 4.75× and 0.713. The SAE beats K-Means encoding on every metric across both blocks (5 seeds, all p ≤ 0.009) and PCA on every metric (all p < 10⁻³).
• The text-embedding baseline is clearly behind on ranking precision. On Open e-commerce it reaches Lift 1.39× on category tasks and 2.13× on life-event tasks, versus the SAE's 1.93× and 5.30×. It achieves Coverage of 1.000 by construction because it embeds every user's history densely.
• On MovieLens, the SAE advantage is structural rather than a ranking win. At matched dictionary size (d_hidden = 1024) and matched L₀, SAE and K-Means encoding are statistically matched on Lift and AUROC (p ≥ 0.1 on both encoders). The SAE instead wins on Coverage (98% versus 81–85% of true positives scored, p < 10⁻⁴ on both encoders), on AUROC_scored for SASRec (p = 0.014, trending on BERT4Rec at p = 0.07), and on reconstruction fidelity (96–97% of variance recovered versus 86–89%).
• The dictionary geometry explains the coverage difference. SAE decoders are near-orthogonal (pairwise cosines centered at 0, mean absolute value 0.06 to 0.08 on SASRec), while K-Means centroids overlap (mean pairwise cosine 0.29 on SASRec, 0.17 on BERT4Rec). Overlapping centroids paraphrase one region of the space and leave other regions uncovered.
• The text baseline collapses on held-out demographics. Because MovieLens ground truth is derived from titles, the authors also test 14 self-reported demographic labels that no stage of the pipeline sees. The text-embedding baseline, which matched the SAE on AUROC for title-derived tasks, falls to near-chance AUROC.
• SAE features are less redundant for audience characterization. Across 20 franchise-level audiences in MovieLens-1M at k = 5, SAE top-k features are significantly less redundant (Stouffer p ≈ 4 × 10⁻⁴) and cover significantly more of the target audience (p ≈ 10⁻⁷). K-Means encoding allocates 2.2× more features per distinct concept (2.20 versus 1.00 redundancy ratio across 5 niche audiences), and 2.5× more on Open e-commerce (2.82 versus 1.14).

Methodology in Plain English

The authors start from a simple model of behavior: each user draws each interaction from a mixture of a small number of latent "behavioral modes," with non-negative weights that sum to one. If the embedding is an average over per-interaction features, the user vector is then approximately a weighted sum of mode centroids. Because those centroids are generally non-orthogonal and can outnumber the embedding dimensions, they sit in superposition, which is exactly the setting where sparse dictionary learning is the right recovery tool.

They use a ReLU sparse autoencoder: an encoder produces non-negative activations penalized by an L1 term, and a decoder with unit-normalized columns reconstructs the input. Each latent is treated as an estimate of a mode weight, and its decoder column as an estimate of the mode direction. The dictionary is deliberately overcomplete (4×), which PCA and ICA cannot be by construction and which NMF cannot match because it requires non-negative input.

Recovery is validated on synthetic data where the true modes are known, so the authors can measure per-feature correlation, per-user cosine alignment between codes and true weight vectors, and the fraction of modes recovered at decoder–centroid cosine ≥ 0.7, with an oracle least-squares ceiling bounding any linear method. They then move to real data: build user embeddings with SVD on category counts (Open e-commerce) or by mean-pooling final-layer hidden states from SASRec and BERT4Rec (MovieLens 1M), train the SAE, and pass each latent's top-activating and non-activating user histories to an LLM to produce a name and description plus a confidence score. At query time an LLM scores each feature's relevance to a natural-language request on a 0–10 scale, features scoring ≥ 5 are kept, and users are ranked by the weighted sum of their activations.

Evaluation uses Lift@n_pos, AUROC, AUROC restricted to scored users, Coverage, and FVU (fraction of embedding variance unexplained). Baselines are K-Means encoding, FastICA, PCA, and a text-embedding baseline built from all-MiniLM-L6-v2 over per-user text summaries. Interpretation uses Claude Sonnet 4.6 and query scoring uses Claude Haiku 4.5, with a Llama 3.3 70B robustness check.

Why This Matters

Impact on research. The paper moves the superposition hypothesis out of transformer residual streams and into a new domain, behavioral embeddings, where ground-truth modes can be constructed and recovery measured directly. It also supplies a counterpoint to recent criticisms of SAEs (Korznikov et al., 2026; Cui et al., 2026) by showing where the frozen-decoder equivalence does and does not hold at moderate overcompleteness (d_hidden/m = 4), and by comparing against K-Means encoding as a strong sparse-decomposition baseline rather than a straw man.

Real-world applications:

• Audience discovery and targeting. Marketers could query "new parents" or "coffee enthusiasts" in natural language and retrieve matching users without seed lists or labeled examples.
• Recommendation transparency and auditing. Mode decompositions give a human-readable account of what a user vector encodes, useful for explaining recommendations or auditing what personal data a system has captured.
• Audience characterization. Given a behaviorally defined segment, the decomposition surfaces a small set of distinct, non-redundant explanatory concepts rather than several paraphrases of the same one.
• Cheap user insights at scale. Because interpretation runs on pre-trained embeddings rather than per-user LLM labeling of raw interaction histories, the cost profile is very different from labeling every user individually.

Industry relevance. The framework is explicitly post-hoc and model-agnostic: it requires no change to the upstream embedding system, no raw interaction data at decomposition time, and no text-aligned training. That makes it deployable on top of an existing production embedding pipeline, which is the practical scenario most personalization teams face.

Future Directions

• Single-position readouts. The linear-pooling argument does not cover last-token or [CLS] embeddings. The authors conjecture that mode resolution emerges from training dynamics because the objective depends on aggregate sequence structure, but they leave verification to future work.
• When attention breaks linearity in the mixture weights. If contextualization makes expected activations non-linear in the mode mix, latents may correspond to finer or composite structure rather than aligning one-to-one with ground-truth modes. Characterizing that regime, and separating attention-driven granularity from artifactual splitting, is open.
• Vocabulary-limited interpretation. Latents whose defining axis falls outside the vocabulary shown to the LLM receive low confidence, which is why retrieval gates at confidence ≥ 0.6. Whether these latents remain monosemantic and interpretable under a richer vocabulary is untested.
• Closing the ranking gap on non-linear encoders. On MovieLens the SAE's advantage shows up in coverage, scored ranking, and reconstruction fidelity but not in raw Lift or AUROC at matched dictionary size. Whether a different sparsity objective or dictionary geometry can convert orthogonality into ranking gains on saturated signals is unresolved.

Target Audience

Researchers and practitioners in mechanistic interpretability who want a non-LLM testbed with ground truth; recommender-system and personalization engineers interested in post-hoc, model-agnostic tools for understanding existing user embeddings; and applied scientists working on audience segmentation, lookalike modeling, or semantic search over non-textual representations. Readers evaluating SAE methodology critically will also find the synthetic ground-truth comparisons and the explicit treatment of the frozen-decoder critique directly relevant.

Authors’ abstract

User embeddings are central components of modern recommendation, personalization, and behavioral modeling systems. Though these embeddings can compactly encode rich behavioral information, they are inherently dense and opaque, limiting interpretability. To address this gap, we argue that user embeddings can be understood as sparse superpositions of interpretable behavioral modes. This structure is directly predicted by generative models of user behavior for linear embeddings and extends approximately to deep-learning representations by the superposition hypothesis. Superimposed modes can naturally be recovered by sparse dictionary learning methods, such as sparse autoencoders (SAEs), and recovered modes can be automatically named and queried in natural language. We validate mode recovery against known ground truth on synthetic data, where SAEs recover modes better than classical sparse baselines, including on transformer-encoded embeddings. We then apply the method to classical and transformer-based user embeddings from two real-world datasets. We also show how the corresponding natural-language labels enable zero-shot audience retrieval and characterization over embeddings never trained on text. The framework is post-hoc and model-agnostic, requiring no modification to the upstream embedding system.