Will the Judge Flip? Predicting Position-Sensitive LLM Judgments from Residual Stream Activations

AI summary

Overview

Research area: LLM-as-a-judge reliability and interpretability — specifically, positional bias in automated judgments and whether internal model activations can predict it.

Technical level: Intermediate. The paper assumes familiarity with transformer residual streams, logistic regression, AUROC/AUPRC, and cross-validation, but its central idea is stated in plain terms.

Scope: A one-sentence summary — the authors test whether an L2-regularized linear probe on a residual stream activation recorded just before an LLM judge's verdict can predict, from the first-pass judgment alone, whether swapping the two candidate responses would change which response the judge prefers.

What This Paper Is About

LLM judges are often used to compare two candidate responses, but their verdicts can depend on which response is shown first. Detecting this sensitivity normally requires judging every pair twice — once in each order — which doubles the cost. This paper asks whether a "flip" can instead be predicted from information already available during the first judgment: the judge's internal activation immediately before it emits its verdict.

Key Contributions

1. A position-flip prediction task defined on a single judgment. The authors formalize a binary label, y_pos = 1[c_AB != c_BA], where c_AB and c_BA are the selected response identities (not presentation slots) under the A-first and B-first orders. The B-first judgment is used only to build the label, never as a probe input.

2. A simple, deliberately minimal predictor. An L2-regularized logistic regression probe is trained on a residual stream activation recorded immediately before the verdict token, at four predeclared relative depths (25%, 50%, 75%, and the penultimate layer), with the layer and regularization strength selected by validation Brier score.

3. A direct comparison against non-activation predictors from the same first-pass judgment. The combined baseline uses six features: verbalized confidence, signed and absolute verdict-logit margin, signed and absolute response-length difference, and the A-first choice itself. All baselines are evaluated on identical held-out folds.

4. A cross-benchmark transfer test. Probes fit on JudgeBench are frozen — no fitting, threshold selection, or recalibration on the target data — and evaluated on MT-Bench comparisons.

Main Findings

• Residual activations predict flips for every judge tested (RQ1). On 534 shared JudgeBench pairs, linear-probe AUROC ranged from .621 (Llama-3.1-8B) to .850 (Qwen3-8B); all four 95% group-bootstrap confidence intervals excluded .5. The full set: Qwen3-1.7B .640 [.595, .686], Qwen3-4B .762 [.719, .804], Qwen3-8B .850 [.815, .884], Llama-3.1-8B .621 [.569, .669].

• Flip rates varied substantially by judge. JudgeBench PFR was .504 for Qwen3-1.7B (269/534 flips), .315 for Qwen3-4B (168/534), .373 for Qwen3-8B (199/534), and .403 for Llama-3.1-8B (215/534).

• The probe beat the combined non-activation baseline (RQ2). AUROC improvements of +.062, +.094, +.074, and +.113 over the combined baseline for the four judges, with every paired confidence interval excluding zero.

• The advantage survived a stricter, choice-matched comparison. Choice-matched AUROC — ranking only between flipping and non-flipping examples that share the same A-first choice — showed improvements of +.108 to +.126, with every paired interval excluding zero. Pooled choice-matched AUROC was .635, .744, .785, and .614 for the four judges, versus .527, .620, .660, and .490 for the combined baseline.

• Not every metric improved uniformly. The paper reports that the Qwen3-4B AUPRC paired interval and the Llama-3.1-8B Brier paired interval include zero, so the paired intervals do not establish improvements on every metric.

• The signal transferred to a different benchmark without adaptation (RQ3). Frozen JudgeBench probes reached AUROCs of .685 [.653, .714] for Qwen3-1.7B, .735 [.691, .778] for Qwen3-4B, .853 [.823, .883] for Qwen3-8B, and .734 [.689, .772] for Llama-3.1-8B on 1,802 shared MT-Bench comparisons. Every interval was above .5.

• Transfer flip rates differed from in-domain rates. MT-Bench PFR ranged from .172 to .379, compared with .315 to .504 on JudgeBench. AUPRC exceeded MT-Bench flip prevalence for every judge, and Brier score was lower than that of a constant predictor fixed to JudgeBench prevalence.

• Repeating the same order changed nothing on the control subset. On a predeclared 100-pair subset, repeating the A-first judgment changed no verdict for any judge, whereas swapping to B-first changed 47 verdicts (Qwen3-1.7B), 19 (Qwen3-4B), 30 (Qwen3-8B), and 43 (Llama-3.1-8B).

• Flips were associated with lower reference agreement. The A-first verdict agreed with the JudgeBench reference on 60–69% of non-flipping pairs but only 47–51% of flipping pairs, which the authors describe as post-hoc accuracies and not evidence that swapping reduces accuracy.

• Retention was incomplete across judges. Of a 620-pair JudgeBench pool, 534 pairs were usable for every judge (86.1%); of 2,400 deduplicated MT-Bench comparisons, 1,802 were usable (75.1%). Per-judge JudgeBench retention ranged from 547 to 590 pairs, and MT-Bench retention from 1,831 to 2,396.

Methodology in Plain English

The researchers collected pairs of candidate responses, each pair paired with a user request. They had four judges — Qwen3-1.7B, Qwen3-4B, Qwen3-8B (all in non-thinking mode) and Llama-3.1-8B-Instruct as a cross-family replication — produce a verdict under a fixed prompt that requests brief reasoning, an integer confidence score from 0 to 100, and a final single-token verdict. Decoding was greedy with a 256-token generation allowance, run in bfloat16 on four 48-GB NVIDIA RTX A6000 GPUs with a 32,768-token context budget in both candidate orders.

Each pair was judged twice, once with A first and once with B first. A "position flip" was recorded whenever the two verdicts selected different response identities. To get the probe input, the authors replayed the saved token sequence through the same token-by-token KV-cache path used during generation and captured the residual stream activation immediately before the verdict token, at four fixed relative depths. They verified that replay reproduced the recorded verdict as the greedy next token for every example in the shared analysis set.

A logistic regression classifier was then trained on those activations to predict the flip label, optimized with deterministic L-BFGS, with the layer and L2 penalty chosen by validation Brier score. Performance was measured with nested grouped cross-validation: five outer groupings, three inner groupings, with all variants of the same question kept together and all fitting and standardization confined to training folds. Predictions from the five outer folds were concatenated to yield exactly one held-out prediction per comparison.

For the transfer test, hyperparameters were selected by grouped cross-validation over all 534 JudgeBench pairs, then a single probe and its standardization were refit on the full JudgeBench set and frozen before being applied to MT-Bench. Reported statistics include position flip rate, AUROC, AUPRC, Brier score, and a choice-matched AUROC, with 95% percentile confidence intervals from 2,000 group-bootstrap resamples using the question as the resampling unit.

Why This Matters

The paper offers evidence that positional sensitivity is partly visible in a judge's internal state before it commits to a verdict, which matters if the goal is to flag unreliable comparisons without paying for a second judgment.

The authors are explicit about the limits of that claim. Probe performance establishes decodability, not causal use — they cite this distinction and interpret their results strictly as predictive evidence. They also state that they do not evaluate a risk-threshold policy, realized judgment savings, or net runtime including activation extraction, and that their linear-probe scores predict position-flip risk rather than general reliability. Untested settings still require evaluating sampled pairs in both orders.

Potential applications, framed as implications of a working risk predictor rather than results demonstrated in the paper:

• Selective re-judging. Spending a second, order-swapped judgment only on pairs the probe flags as high-risk, rather than on every pair.
• Benchmark auditing. Screening large comparison sets for candidate-order sensitivity before publishing model rankings.
• Judge selection and monitoring. Comparing judges by how much of their flip behavior is linearly decodable from pre-verdict activations, as one signal alongside PFR.
• Evaluation cost planning. Informing how much of a judgment budget is likely to be consumed by re-checking if flips are concentrated in a predictable subset.

Industry relevance: Any pipeline that uses an LLM to compare outputs — model leaderboards, RLHF or preference-data generation, regression testing of model releases, and automated evaluation in production — has a direct interest in when an order swap changes the answer. The paper's practical constraint is that residual stream access excludes closed models; the authors state this limitation directly.

Future Directions

1. Test the causal question the paper explicitly leaves open. The authors note that linear decodability does not establish that the model uses the decoded information causally or explicitly estimates positional susceptibility. Interventional experiments would be the natural next step.

2. Move beyond a linear probe and a fixed feature set. The paper notes that outperforming the fitted linear baseline does not exclude nonlinear predictive relationships among its features, and that the combined baseline does not exhaust all measures of difficulty or uncertainty.

3. Extend to other judges, data, and prompts. The study covers four relatively small open-weight judges, including three from the same Qwen3 family, on two response-pair distributions. PFR may depend on judge, data, or prompt, and the authors' evaluation prompt differs from the official FastChat templates, precluding comparison with official MT-Bench estimates.

4. Evaluate the operational side. A risk-threshold policy, realized judgment savings, and net runtime including activation extraction were not evaluated. The retention constraint also matters: the shared analysis sets required valid outputs in both candidate orders for every judge, so results are conditional on format compliance and may not generalize to excluded comparisons.

Target Audience

Researchers and practitioners working on LLM-as-a-judge systems, evaluation reliability, and model interpretability will get the most from this paper. It is also relevant to engineers building automated comparison or preference-generation pipelines who need to reason about the cost and value of double-judging, and to methodologists interested in what linear probing of residual streams can and cannot demonstrate. Readers should come with working knowledge of AUROC, cross-validation design, and transformer internals; the paper's mathematical framing is compact and the appendix tables carry much of the quantitative detail.

Authors’ abstract

The order in which candidate responses are presented can change an LLM judge's verdict. Detecting such a position flip ordinarily requires judging each pair in both orders, which doubles the number of judgments. We investigate whether residual stream activations recorded immediately before the initial verdict can predict a flip. We use nested grouped cross-validation to evaluate regularized linear probes on 534 JudgeBench pairs for three Qwen3 judges and Llama-3.1-8B. The linear probes achieve AUROCs of .621-.850 and outperform a combined baseline that uses verbalized confidence, verdict-label logits, response lengths, and the judge's initial choice by .062-.113 AUROC. Linear probes trained on JudgeBench and then frozen achieve AUROCs of .685-.853 on 1,802 MT-Bench comparisons without MT-Bench fitting or recalibration. These results show that pre-verdict activations support prediction of susceptibility to candidate order and outperform the non-activation predictors evaluated here.