LLM Probing with Contrastive Eigenproblems: Improving Understanding and Applicability of CCS

AI summary

Overview

• Research area: Mechanistic interpretability of large language models, specifically unsupervised linear probing of LLM internal activations (contrast-consistent probing for binary features such as sentence truth).
• Technical level: Advanced. The paper works with eigendecompositions, covariance/cross-covariance matrices, singular value decompositions, and formal geometric accounts of linear probes.
• Scope in one sentence: The paper dissects the two-term loss of Contrast-Consistent Search, argues that contrast consistency should be defined relatively, and reformulates the objective as a closed-form contrastive eigenproblem (Difference-Relative Contrast and Ratio-Relative Contrast) that matches CCS accuracy while removing sensitivity to random initialization.

What This Paper Is About

Contrast-Consistent Search (CCS), introduced by Burns et al. (2023), is an unsupervised probe that asks whether an LLM's hidden states encode a binary feature such as sentence truth. It does so without human labels, using contrastive pairs (a sentence and its negation) and a two-term loss: a consistency loss that forces the probe's probabilities for a pair to sum to one, and a confidence loss that was originally motivated as a way to avoid the degenerate solution where every input receives probability 0.5.

The authors show that this stated rationale for the confidence term is incomplete: the confidence term is doing most of the work, and its real effect is to bias the learned direction toward high-variance directions such as the leading principal components. They argue the right target is relative contrast consistency, and derive closed-form solutions to the resulting eigenproblem.

Key Contributions

1. A loss-term ablation of CCS. The authors ablate the consistency and confidence terms separately, and also test two modifications (restricting probes to unit vectors, and removing the null space via SVD) intended to defeat the degenerate solution without the confidence term.

2. A reframing of what the confidence loss actually does. The paper shows empirically and analytically that the confidence loss biases the probe toward high-variance, principal-component-like directions, and argues CCS should instead optimize relative contrast consistency (RCC).

3. A closed-form reformulation as contrastive eigenproblems. CCS is recast into Difference-Relative Contrast (DRC) and Ratio-Relative Contrast (RRC), yielding closed-form solutions, interpretable eigenvalues, and no dependence on random initialization.

4. Eigenvalues as a diagnostic, plus a multi-feature extension. The eigenvalues indicate how well a dataset isolates a single contrastive feature, and the method extends naturally to multiple variables, demonstrated by replicating the Bürger et al. (2024) result that truth and polarity are encoded in a shared subspace.

Main Findings

• The confidence loss does more than prevent the degenerate solution. Ablating the confidence term reduces accuracy more than ablating the consistency term, and the two proposed alterations (unit-vector constraint, SVD null-space removal) do not compensate for its removal. For example, on the comparisons dataset, CCS reaches 100 ± 00 and confidence-only reaches 100 ± 01, while consistency-only variants land at 66 ± 11, 62 ± 09, 65 ± 11 and 59 ± 07.

• Alterations alone are harmless. Adding the unit-vector and SVD modifications to the full CCS objective (CCS+a1+a2) preserves accuracy: 100 ± 00 on comparisons, 99 ± 00 on sp_en_trans, 99 ± 00 on cities, 94 ± 00 on amazon, and 87 ± 01 on imdb.

• The confidence loss pushes probes toward the leading principal components. Measured by the extent to which the learned vector lies in the subspace spanned by the first K principal components, confidence-only probes reach an average cosine similarity of over 0.5 with the first principal component on the IMDB dataset using answer-token activations, averaged over 30 random seeds. Consistency-only probes have almost no magnitude in that high-variance subspace.

• CCS is not uniformly reliable across datasets. In layer 16 of Llama-2-7b, with 30 seeds, some datasets reach stable high accuracy (comparisons at 100 ± 00 on the answer token, cities at 99 ± 00, sp_en_trans at 99 ± 00 on the answer token and 100 ± 01 on the period token, amazon at 93 ± 00) while others are weak or unstable (copa at 59 ± 06 on the answer token and 48 ± 02 on the period token, rte at 54 ± 06 and 54 ± 07, snli at 71 ± 14 and 85 ± 06, ent_bank at 79 ± 10 and 82 ± 16, imdb at 87 ± 00 and 87 ± 07).

• The new eigenproblem methods match CCS without initialization sensitivity. The paper reports accuracy for DRC and RRC alongside CCS (with minimum, median and maximum across seeds), and finds that DRC and RRC recover similar performance while avoiding the problems around sensitivity to random initialization. Datasets such as copa and rte remain difficult across all methods.

• The two geometries coincide. Formulating relative contrast consistency as an eigenproblem of differences in variance forces the assumption that the hyperplane normal n and the translation direction t are the same. Both the DRC consistency and displacement versions reduce to the same two cross-term matrices, and thus yield the same bases in opposite order.

• Datasets that fail CCS fail to isolate a single feature. The eigenvalues make this visible: large negative eigenvalues (for the commonality matrix) and large positive eigenvalues (for the displacement matrix) indicate a dataset that isolates one contrastive feature well, while poor eigenvalues flag datasets where CCS is unreliable.

• Multiple features can be probed at once. The approach reproduces the Bürger et al. (2024) finding that truth and polarity are encoded together in a shared subspace, demonstrating that contrastive eigenproblems extend beyond single binary features.

Methodology in Plain English

The authors start from the observation that CCS has a two-part objective, and they test which part matters by training probes with each part switched off, in various combinations. They also add two fixes intended to prevent the trivial "everything is 50/50" answer: forcing the probe vector to have unit length, and projecting the data onto its non-degenerate subspace using an SVD so the probe cannot hide in directions where the data has no variation.

They then ask what the confidence term is mathematically doing. Since the confidence term is minimized when probabilities are extreme, it is effectively pushing the probe direction to align with inputs that have large projections — in other words, high-variance directions. To test this, they measure how far the learned probe vector reaches into the subspace spanned by the first K principal components, comparing full CCS, confidence-only, and consistency-only probes.

Their diagnosis is that absolute consistency is not the right target: a direction can look perfectly consistent simply because the hidden states barely vary along it. What matters is consistency relative to how much the representations actually vary. They formalize this by defining two matrices over contrast pairs: the commonality matrix (the sum of positive and negative activations, capturing what the pair shares) and the displacement matrix (the difference, capturing what changes between the pair). Finding a direction that shrinks variance in the commonality matrix, or grows it in the displacement matrix, relative to the original data is a (generalized) eigenvalue problem, which has closed-form solutions. Finally, they evaluate these closed-form directions against CCS on the same datasets, tokens, model, and layer.

Why This Matters

Impact on research. CCS has been used as evidence about what LLMs internally represent, but it has also been criticized for instability and for finding arbitrary directions. This paper explains why those failures happen — the confidence term is silently selecting high-variance directions — and replaces a gradient-descent-plus-random-seed procedure with a deterministic, closed-form solution. The eigenvalues also give a usable diagnostic: if the leading eigenvalue does not stand out, the dataset probably does not isolate a single feature, which is a reason to distrust the probe before running it. Because the formulation extends to multiple features, it opens a path to probing several variables at once rather than one at a time.

Real-world applications (the paper itself reports no deployed application; these are plausible implications of the method):

• Hallucination and calibration checks: probing whether a model's internal states separate true from false statements could support uncertainty or reliability signals.
• Honesty, deception and safety auditing: an unsupervised probe that does not assume the model agrees with human labels is a natural tool for auditing what a model internally treats as true.
• Evaluation beyond benchmarks: the eigenvalue diagnostic could be used to judge whether a probing dataset is informative before it is used in an evaluation claim.
• Causal intervention and steering: the displacement-style direction is the kind of direction used to intervene on a model (via activation addition, as in Turner et al., 2024), for example to change whether a model treats a statement as true.

Industry relevance. Teams that want to make claims about what a model "knows" or "believes" currently rely on probes that must be retrained with many random seeds and checked for variance. A closed-form solution removes a source of nondeterminism from that pipeline, and the eigenvalue diagnostic provides a cheap pre-check on whether a given dataset supports the claim being made.

Future Directions

• Reconciling classification-style and intervention-style probes. DRC forces n = t; the authors note this means the eigenproblem formulation cannot distinguish the separating hyperplane normal from the translation direction. Finding a formulation that separates these two geometries remains open.
• More than two variables. The paper extends to multiple features and replicates the truth-and-polarity result; scaling to larger sets of interacting features is a natural next step.
• Using eigenvalues as a dataset-quality filter. The observed link between poor eigenvalues and datasets where CCS is unreliable invites systematic study of which contrastive datasets fail to isolate a feature, and why.
• Broader probing methods. The authors present relativized contrast consistency as a pathway to probing and mechanistic interpretability methods beyond CCS, which could be explored for non-binary or continuous features.

Target Audience

Researchers in mechanistic interpretability and LLM representation analysis who are familiar with probing methods and linear algebra at the level of eigendecompositions and SVD. It is most valuable to readers who already know CCS and want to understand its failure modes, and to practitioners who use unsupervised probes for truthfulness or feature detection and need a deterministic, diagnostic alternative to gradient-descent training over many seeds. It is not an introductory paper; the derivations assume comfort with linear algebra notation.

Authors’ abstract

Contrast-Consistent Search (CCS) is an unsupervised probing method able to test whether large language models represent binary features, such as sentence truth, in their internal activations. While CCS has shown promise, its two-term objective has been only partially understood. In this work, we revisit CCS with the aim of clarifying its mechanisms and extending its applicability. We argue that what should be optimized for, is relative contrast consistency. Building on this insight, we reformulate CCS as an eigenproblem, yielding closed-form solutions with interpretable eigenvalues and natural extensions to multiple variables. We evaluate these approaches across a range of datasets, finding that they recover similar performance to CCS, while avoiding problems around sensitivity to random initialization. Our results suggest that relativizing contrast consistency not only improves our understanding of CCS but also opens pathways for broader probing and mechanistic interpretability methods.