The Linear Representation Hypothesis Needs a Group Action

AI summary

Overview

Research area: Interpretability and representation learning theory — specifically the mathematical foundations of the Linear Representation Hypothesis (LRH), drawing on group theory, linear algebra (dual spaces, projective spaces, Grassmannians), and architectural symmetry.

Technical level: Advanced. The paper is formal and assumes comfort with group actions, equivariance, dual vector spaces, and projective geometry. It contains no experiments.

Scope: The paper argues that any claim about a model's internal representations must explicitly specify a representation space with a group action, an equivariant estimation procedure, and an invariant predicate, and it uses this specification to audit common interpretability quantities and analyses.

What This Paper Is About

Interpretability research routinely asks whether a model "linearly represents" a feature, but rarely states when two representations should count as the same. Without that, a cosine similarity, a probe, and a steering intervention can look like they study one representation when they actually assume different equivalences. The paper's goal is to formalize representation equivalence with group actions and show what changes when the choice is made explicit.

Key Contributions

1. A formal definition of a representation claim. A claim is a quadruple (G, M, F, P): a group G acting on representations, a G-space M holding the extracted object, an equivariant procedure F producing it, and an invariant predicate P asserted of it. Admissibility requires (A1) F equivariant and P invariant, and (A2) G containing the equivalences the architecture itself imposes.

2. The argument that the LRH is a family, not one hypothesis. Displacement, linear decodability, superposition, and subspace formulations place features in different mathematical objects — V, the dual V*, affine decompositions, ℙ(V), and Gr(k, V) — with different transformation laws and different admissible groups.

3. An architectural floor and a reading-point result. Function-preserving parameter reparameterizations lower-bound the admissible G, and the induced symmetry differs by where the representation is read (residual stream versus query or key sites).

4. A symmetry audit and a reporting principle. A table classifies common interpretability quantities by the largest group under which they are preserved within the hierarchy G_iso ⊂ G_sim ⊂ G_aff, and the paper applies the same audit to the Platonic Representation Hypothesis.

Main Findings

• "Linear representation" does not pick out a unique object. A displacement lives in V and transforms as v ↦ Av under an affine map h ↦ Ah + t, since the translation cancels. A linear probe lives in the dual V* and transforms as w ↦ A⁻ᵀw, with a bias extended by b ↦ b − w(t). Both are stored as d-dimensional arrays but obey different laws.

• Primal and dual objects cannot be identified without extra structure (Proposition 3.2). For dim V ≥ 2, there is no nonzero GL(V)-equivariant map V → V*. An inner product induces an equivariant map under G_iso only; passing to projective directions enlarges the symmetry to G_sim. Regularized ℓ2 probe fitting breaks equivariance under general GL(V), leaving only isometric equivariance.

• The metric enters through the evidence, not the hypothesis. None of the LRH formulations intrinsically requires an inner product, yet cosines and spectral quantities require similarity geometry while absolute norms and intervention magnitudes require a fixed scale.

• Architectural floor (Proposition 4.1). If a function-preserving parameter transformation γ (from the group Γ of invertible reparameterizations with f_{γ·θ} = f_θ) induces ρ(γ) at a reading point and changes the value of P(F(φ)), that claim is a property of the parameterization rather than of the model function. Hence G must contain Im ρ.

• Reading-point dependence (Corollary 4.2). In dot-product attention, W_Q ↦ ΛW_Q and W_K ↦ ΛW_K preserve the model function for any Λ ∈ GL(d_head). At the residual stream this acts trivially and excludes no angular claim; at a query or key site Im ρ contains GL(d_head), which admits no nonzero invariant bilinear form — taking Λ = cI forces b = 0. Angular and metric predicates built from a fixed inner product are therefore not invariant at those sites, and RoPE reduces but does not remove the anisotropy.

• Composition matters (Proposition 4.3). A pipeline is limited by its most restrictive stage. A direction estimated under G_aff, compared by cosine under G_sim, and calibrated by a norm under G_iso is guaranteed admissible only under G_iso unless stronger invariance of the composite is established. Symmetry of a composite is not obtained by intersecting the groups of stages considered in isolation.

• Symmetry audit of common quantities. Preserved under G_aff: exact rank, span, containment and intersection dimension, CCA. Preserved under G_sim: leading principal subspace, relative-threshold rank, effective rank and participation ratio, intrinsic dimension, cosine/angle/orthogonality, principal angles, Grassmann distances, linear CKA, nearest-neighbour agreement. Preserved under G_iso: norm and distance, intervention magnitude, absolute-threshold rank, Procrustes distance, reconstruction loss. Preserved under signed permutations of latent coordinates: coordinatewise sparsity penalty.

• Dictionary learning restricts two spaces separately. Under h ↦ Ah + t with D ↦ AD, W_enc ↦ W_enc A⁻¹, and biases absorbing t, encoder preactivations and latent codes are unchanged; the Euclidean reconstruction loss is preserved for all residuals only when AᵀA = I, which forces G_iso on the activation side. On the latent side, W ↦ WB and x ↦ B⁻¹x leave reconstruction unchanged, while a coordinatewise sparsity penalty reduces this to signed permutations.

• Practice leaves the specification implicit. The paper documents identification by storage format (probe weights and steering vectors both used as additive intervention directions and compared by cosine), normalization and calibration conventions that differ between Contrastive Activation Addition and Activation Addition (the same numerical coefficient does not denote the same displacement), unnamed reading points (geometric quantities reported in per-head key or query space are not invariants of the computation they explain), and inconsistency within a single analysis.

• Platonic Representation Hypothesis inherits the same omission. Different-width models need not share a representation space, so comparison proceeds through induced structures such as neighbourhoods or similarity matrices and is indexed by the input distribution used. On a finite sample, when representation width is at least the number of sampled inputs and both activation matrices have full row rank, a similarity measure invariant to arbitrary invertible linear transformations cannot distinguish them, so finite-sample convergence can become near-trivial in that regime. Existing evaluations operate closer to G_sim, with the effective equivalence set by the alignment measures in use.

• Admissibility alone carries no content; maximality does. Condition (A1) can always be satisfied by shrinking G or weakening P, so the strength of a claim is the largest group under which it survives. Declaring G_iso throughout is forbidden by (A2) whenever the architecture realizes transformations outside G_iso.

Methodology in Plain English

The paper is theoretical. It begins by observing that equivalence is unspecified in existing LRH discussions, then builds a vocabulary: representations are functions φ from an input domain into a vector space, a group G acts pointwise on representation values, and equivalence means membership in the same orbit. Within the affine class it defines a hierarchy of groups — G_aff (arbitrary invertible linear part plus translation), G_sim (linear part sQ with s > 0 and Q orthogonal), and G_iso (s = 1) — and describes what each progressively preserves.

It then separates the object being studied from the procedure that produces it, requiring the procedure to be equivariant and the predicate invariant, rather than only requiring the composite to be invariant. This separation is what lets the paper distinguish a displacement in V from a probe in V* even though both are stored identically.

Two constraints follow. Externally, the group of function-preserving parameter transformations Γ induces a symmetry at each reading point, and admissible claims must respect it. Internally, composition of analysis stages must be checked as a whole. The paper then audits a list of standard interpretability quantities against this framework, classifying each by the largest group preserving it, and applies the same audit to case studies of published analyses and to the Platonic Representation Hypothesis. No experiments, datasets, benchmarks, or numerical results are reported.

Why This Matters

Impact on research. The framework turns vague disputes about whether two studies "found the same thing" into a checkable question: do they use the same representation equivalence and the same G-space? It explains why equal numerical shape — two length-d arrays, or two residual streams of the same dimension — is insufficient grounds to identify objects, and it gives a criterion for when a reported geometric property belongs to the model rather than to a particular parameterization.

Real-world applications:

• Steering and safety interventions. Refusal-direction methods and activation-addition methods consume directions in different spaces and under different normalization conventions; the framework makes explicit which calibration is being assumed and what a reported direction magnitude actually measures.
• KV-cache eviction and attention analysis. Methods such as KeyDiff evict entries using pairwise cosine similarity among keys, and report key and query cosines, norms, PCA of key caches, and log det(K Kᵀ) in per-head key or query space. The paper argues those quantities are not invariants of the attention computation they are used to explain.
• Model comparison and alignment evaluation. Cross-model convergence claims in the Platonic Representation Hypothesis tradition depend on an unspecified equivalence; the framework identifies when such claims become near-trivial on finite samples.
• Probing and dictionary-learning pipelines. The paper shows how preprocessing such as singular-value truncation in SVCCA destroys the affine invariance of a subsequent CCA stage, and how the reconstruction loss and the sparsity penalty restrict the activation space and the latent space separately.

Industry relevance. Teams that train probes, fit sparse autoencoders or dictionaries, extract steering vectors, or report head-level geometry can use the reporting principle to state assumptions that currently have to be reverse-engineered from code. The paper is explicit that this has a cost: some conclusions become narrower, some comparisons require an additional map between spaces, and some quantities must be attributed to a representation together with a chosen metric or parameterization.

Future Directions

• Discharge the empirical obligation. The paper notes that a property varying across Im ρ is a property of θ together with the producing procedure, not of f_θ, and that asserting it requires showing the reported geometry is stable across seeds and training runs — something rarely done.
• Fix the equivalence for cross-model convergence. The truth value of the Platonic Representation Hypothesis turns on the transformation group, which remains open; existing evaluations operate closer to G_sim, with the effective equivalence set by the alignment measures rather than specified independently of them.
• Determine which inner product supports a given predicate. The paper frames the disagreement over near-orthogonality of concepts — given existing arguments that such near-orthogonality can arise generically in high dimensions and be induced by whitening — as a precise question about the inner product rather than a mere relabeling of assumptions.
• Supply explicit equivariant maps between reading points and between G-spaces. Equal-dimensional representations at different reading points may carry different actions, so transporting a direction between them requires such a map rather than identification by shared coordinates.

Target Audience

Researchers and advanced practitioners in mechanistic interpretability and representation learning who build or evaluate probes, steering vectors, sparse autoencoders, dictionary-learning methods, or representation-similarity metrics, and who are willing to engage with group-theoretic formalism. It is also relevant to theorists working on cross-model representation comparison, and to engineers who need a vocabulary for specifying what a reported direction or geometry is actually a property of. Readers seeking empirical results, benchmarks, or implementation recipes will not find them here.

Authors’ abstract

To make claims about representations that generalize beyond a particular trained model, we need to specify when two representations should count as equivalent. The Linear Representation Hypothesis is often discussed without making this equivalence explicit. Different notions of equivalence preserve different structures, so metrics, probes, and interventions that appear to study the same representation may in fact correspond to different hypotheses. We therefore argue that the Linear Representation Hypothesis is not one hypothesis but a family of claims distinguished by representation equivalence. We formalize this idea using group actions, specifying the representation object, the procedure that produces it, and the property ultimately asserted, while accounting for equivalences imposed by the model architecture. This framework clarifies how assumptions can change across metrics, reading points, and analysis stages, and we use it to audit common representation quantities and recent interpretability analyses.