U-Space: Uncovering When and Why Uncertainty Arises in Language Models

AI summary

Overview

Research area: Uncertainty quantification for large language models, combined with mechanistic interpretability.

Technical level: Advanced. The method builds on Jacobian-based lens techniques, residual-stream geometry, orthogonalization, and activation steering.

Scope: The paper introduces an interpretable, training-free subspace of a language model's residual stream that reveals where and which kind of uncertainty arises during reasoning, and shows that a scalar score read from it outperforms established uncertainty baselines under both standard and length-controlled evaluation.

What This Paper Is About

Uncertainty estimators tell you how confident a model is, but usually not why it is uncertain or where in a reasoning trace that uncertainty entered. Many existing methods also need repeated generations, separately trained components, or correctness labels, and recent work shows that generation length alone is a strong predictor of correctness, raising the question of how much of an estimator's power is genuine uncertainty information rather than a length shortcut. The paper's goal is to build an uncertainty signal that is interpretable, requires no correctness labels, no repeated sampling, and no training, and that remains predictive after controlling for output length.

Key Contributions

1. A training-free recipe for interpretable contrastive directions. The authors construct semantic directions in the residual stream from small sets of uncertainty and certainty anchor words, instantiated as the U-Space, which reveals when uncertainty arises and what form it takes. The construction uses no task examples, correctness labels, or fitted parameters.

2. The first estimator to jointly operationalize first- and second-order uncertainty. The U-Lens combines mean predictive entropy (first-order, distributional) with a verbalizable cone-alignment signal (second-order) while retaining a token-level semantic account of where and why uncertainty arises.

3. Empirical gains across models and benchmarks. Across three reasoning models and four benchmarks, the trace-level score outperforms the compared baselines under both standard and length-controlled evaluation and transfers more reliably than the supervised baseline.

4. Causal evidence through intervention. Steering along the U-Space basis directions can make the model hesitate and lose confidence even on simple questions, indicating the directions capture representations that shape behavior rather than merely correlate with it.

Main Findings

• Generation length is a strong shortcut in unadjusted evaluation. Length alone reaches 68.6% ± 1.4 AUROC, nearly matching the supervised Feature-Gaps baseline (68.8% ± 4.3) and trailing the five-pass TokUR (69.6% ± 4.9) by only 1.0 point. Under length control, generation length falls to 52.4% AUROC, and several established estimators decline with it (TokUR by 9.4 points, SentenceSAR by 5.9, Feature-Gaps by 5.5, Maximum Entropy by 5.0).

• The U-Lens is the only method to lead all nine model-metric comparisons in the length-controlled table. Averaged over benchmarks and seeds, it reaches length-controlled AUROC of 68.4 ± 0.4 on Gemma 4, 71.0 ± 0.7 on Qwen3.5, and 68.3 ± 1.0 on Magistral 1.1.

• The U-Lens degrades least under length control. Its unadjusted-to-length-controlled AUROC drop is only 1.8 points, compared with 16.2 for generation length and 9.4 for TokUR.

• Gains over the strongest baseline in the length-controlled setting. AUROC improves by 2.2, 1.9, and 4.7 points on Gemma 4, Qwen3.5, and Magistral 1.1 respectively. AUPRC improves by 0.8 to 4.2 points, and AURC is reduced by 0.4 to 2.5 points.

• Extra compute does not buy robustness. After controlling for length, the additional generations required by several estimators no longer translate into stronger performance: P(True), TokUR, and SentenceSAR fall below predictive entropy in all but one of the nine comparisons, while Semantic Entropy exceeds it by only 0.1 points on the cross-model average.

• The two components are complementary and their combination is consistently better. On Gemma 4, predictive entropy and the cone-alignment score trade the advantage across metrics; predictive entropy is stronger throughout on Qwen3.5, while cone alignment is stronger throughout on Magistral 1.1. Their product in the U-Lens improves over both components on all three models.

• Models internally separate the four uncertainty categories. Contrasting hidden states from minimal prompt pairs shows that J-Lens decoding of the ambiguity direction promotes ambiguous and ambiguity, conflicting evidence promotes controversial and contentious, incompleteness promotes cannot and unable, and general uncertainty promotes dubious, suspicious, and questionable.

• Token-level readouts trace an interpretable path. In one Qwen3.5 answer about Hamlet's handedness and his love for Ophelia, tokens exhibit incompleteness while the model states what Shakespeare left unsaid, tip to ambiguity at play, turn to conflicting evidence as the answer affirms Hamlet's love against his cruelty towards Ophelia, and end in general uncertainty at the closing her.

• The directions are causally relevant. Adding a category direction to the residual stream over a contiguous span of tokens and layers during prefill or reasoning changes model behavior; steering along ambiguity during reasoning causes the model to lose confidence in the targeted span.

• The construction is not limited to uncertainty. Preliminary evidence in the appendix applies the same label-free recipe to emotion classification and reward-hacking detection.

Methodology in Plain English

The authors start with short lists of words that express doubt (uncertainty anchors) and words that express the matching kind of certainty, drawn from prior word lists and filtered down and assigned to four categories: ambiguity, incompleteness, conflicting evidence, and general uncertainty.

To turn a word into a direction inside the model, they work backwards. A word's direction in the output vocabulary is pulled back through an averaged Jacobian — a precomputed map of how intermediate states influence the final layer — combined with the model's unembedding matrix and its final normalization gain. This is the J-Lens pullback. For models with no public lens, they estimate the Jacobian from unlabeled, generic prompts without optimization or parameter updates.

For each category, they average the pulled-back uncertainty anchors and the pulled-back certainty anchors separately, normalize both, and subtract the certainty centroid from the uncertainty centroid. This removes shared components and orients the resulting vector toward uncertainty. Stacking the four category vectors and orthonormalizing them gives the U-Space basis.

To read uncertainty, they project the normalized hidden state of every reasoning token onto this basis, producing a signed coordinate per category. They rectify those coordinates into the positive cone using a softplus with a fixed gain of the square root of the number of categories, then take the length of the resulting vector. This score grows with alignment to any positive combination of uncertainty directions, so being negative on one category cannot cancel being positive on another. Normalizing before projecting discards overall projection strength and keeps only orientation.

Rather than pooling this over a variable number of reasoning tokens, they evaluate it once at the end-of-thinking marker, which is conditioned on the full trace. They multiply this verbalizable signal by the mean predictive entropy computed over the reasoning tokens. Averaging rather than summing entropy avoids mechanical scaling with trace length. No correctness labels, repeated generations, or task-specific training are needed.

For evaluation they use three open-weight reasoning models — Gemma 4 (Gemma-4-31B-it), Qwen3.5 (Qwen3.5-27B), and Magistral 1.1 (Magistral-Small-2507) — reading from a layer two-thirds of the way through the network (layer 40, 42, and 27 respectively) to avoid dataset- or model-specific layer selection. They evaluate on MMLU-Pro, Omni-MATH, SuperGPQA, and TriviaQA, each with a fixed seeded stratified sample of 1,000 examples, over three seeds. Metrics are AUROC, AUPRC, and AURC, reported both unadjusted and length-controlled. For length control, outputs are split into ten quantile bins by token count, each metric is computed within each bin, and the results are averaged weighted by bin size.

Why This Matters

Impact on research. The paper reframes uncertainty quantification as an interpretability problem: instead of compressing everything into one number, it exposes which category of uncertainty is active at each token and how the mix evolves. It also contributes a length-controlled evaluation protocol and demonstrates that multi-pass methods do not automatically hold up under it, which is a challenge to a common assumption in the field. The label-free construction is shown to be transferable to non-uncertainty semantics, suggesting a general recipe for building interpretable concept bases.

• High-stakes assistants: flagging individual answers that should be deferred to a human expert, which the paper frames as the central motivation when the cost of error exceeds the cost of deferral.
• Reasoning and mathematics tutoring: surfacing the exact step in a derivation where the model became uncertain, rather than only marking the final answer as suspect.
• Model auditing and monitoring: inspecting token-level category maps and using controlled interventions to probe how a model represents its own reliability.
• Factual question answering: producing per-token uncertainty attribution for long-form answers where a single unreliable claim can hide inside an otherwise correct response.

Industry relevance. The method works on a single generation with no correctness labels and no training, which keeps inference cost low relative to multi-pass estimators like TokUR (five passes), Semantic Entropy (ten sampled answers), or P(True) (a second pass). That combination of low cost and interpretability is attractive for deployed systems that need per-answer gating decisions rather than aggregate accuracy numbers. The authors caution, however, that the readout is not a calibrated safety detector and should not replace independent verification in consequential settings, and that because the same directions can alter model commitment, deployments must evaluate both reliability and potential misuse.

Future Directions

• Broaden the taxonomy. The authors state explicitly that the four categories are not intended to be exhaustive; richer or domain-specific category sets could be defined by swapping the anchor sets.
• Extend beyond uncertainty. Only preliminary appendix evidence exists for emotion classification and reward-hacking detection, so the generality of the label-free construction remains an open question.
• Resolve the component complementarity. Predictive entropy and cone alignment each win on different models, and there is no reported analysis of why; understanding this could lead to better combination rules than the fixed multiplicative form.
• Quantify calibration and robustness. The paper reports AUROC, AUPRC, and AURC but not calibration metrics, and it leaves open how sensitive the readout is to the choice of layer, anchor lists, and Jacobian approximation for models without precomputed lenses.

Target Audience

Researchers and engineers working on LLM reliability, hallucination and error detection, and uncertainty quantification, particularly those who need per-answer confidence without extra inference passes. It is also relevant to mechanistic interpretability researchers interested in connecting verbalizable concepts to residual-stream geometry, and to practitioners building evaluation pipelines who now need to account for generation length as a confound. The paper assumes familiarity with residual streams, unembedding matrices, entropy, and AUROC, so it is not suited to readers without a machine learning background.

Authors’ abstract

Large language models are informing decisions with ever-higher stakes. As the consequences of their errors grow, a central question becomes harder to ignore: how much can we trust an individual answer? Yet recognizing when to defer remains difficult because language models can present incorrect conclusions with fluent explanations and an authoritative tone. Uncertainty quantification seeks to address this disconnect by estimating the reliability of individual predictions. However, many existing methods require repeated generations or separately trained components, and their scalar estimates do not reveal where uncertainty arises or how it evolves during reasoning. Recent work has also shown that generation length can be strongly associated with uncertainty estimates and correctness, raising the question of how much of an estimator's predictive power comes from uncertainty-specific information rather than output length alone. Mechanistic interpretability offers a way to address these limitations by connecting human-interpretable concepts to intermediate model states. Building on this capability, we introduce the U-Space, a low-dimensional subspace that makes a model's evolving uncertainty measurable and interpretable. We identify semantic anchors for doubt and certainty, map their unembedding directions back into the residual space, and combine their contrasts into an orthogonal basis. The U-Lens projects each token state onto these basis vectors, yielding an interpretable token-level uncertainty map that can be inspected directly or aggregated into a scalar uncertainty score. Our approach requires no correctness labels, repeated generations, or training. Across reasoning benchmarks, its confidence score outperforms established baselines under both standard and length-controlled evaluation and transfers more reliably than supervised estimators. Code: https://github.com/s2labres/U-Space.