On the Limits of Test-Time Compute for Self-Evaluation

AI summary

Overview

Research area: Large language model reasoning and test-time compute, specifically self-verification, tree search, and solution aggregation. Technical level: Intermediate. The paper uses a formal bias–variance decomposition of a pairwise ranking loss, but the core ideas and the proposed method (Scorpio) are explained with a modest amount of mathematics and are accessible to readers familiar with LLM sampling and scoring. Scope: The paper diagnoses why spending more inference compute on self-evaluation fails to close the gap between pass@1 and pass@N, and proposes a simple random-group scoring method, Scorpio, that improves verification, search, and aggregation.

What This Paper Is About

Sampling many candidate solutions raises the chance that at least one is correct (the pass@N oracle), but models are poor at picking out the correct one, so selection accuracy plateaus well below pass@N. The authors measure this failure as a pairwise ranking error — how often a model scores an incorrect solution above a correct one — and show that this error splits into a bias term (pairs the model consistently misranks) and a variance term (inconsistency across repeated judgments). Their goal is to understand which part of the error additional compute can fix, and to design a scoring method that reduces the part that compute can address while keeping the rest low.

Key Contributions

1. A bias–variance decomposition of ranking error for self-evaluation. Adapting the pairwise ranking loss from Cortes and Mohri (2003) and the decomposition of Shivaswamy and Chandrashekar (2021), the authors show that expected ranking error equals squared bias plus variance, with each correct–incorrect pair contributing p_ij^2 and p_ij(1 - p_ij), where p_ij is the probability of misranking that pair.
2. An empirical account of how inference compute and algorithm design shift bias and variance. Across math, code, and puzzle benchmarks and two open models, they find that variance generally falls with compute while bias decreases initially and then plateaus, leaving a residual bias that more compute does not remove. They estimate that bias accounts for 91% to 99% of ranking error as compute grows.
3. Scorpio (Scoring Candidates Over Random Partitions). A simple, training-free scoring procedure that repeatedly partitions a candidate pool of size N into random groups of size K, scores each group jointly in one LLM call, and averages each candidate's ratings over M rounds, using MN/K LLM calls in total.
4. Evidence that Scorpio improves three distinct test-time scaling settings — verification, tree search, and aggregation — across multiple models and benchmarks, with reported gains of up to 7 percentage points in verification accuracy, up to 10 percentage points in tree search on LiveCodeBench v6, and 11 percentage points for aggregation on coding tasks.

Main Findings

• The selection gap is real even for frontier models. On Terminal-Bench 4, GPT-6 Astra gains 13.0 points and Fable 5.1 gains 20.9 points moving from pass@1 to pass@5. The gap between selection accuracy and pass@N widens as N increases for V1 (Singh et al., 2026) and for Recursive Self-Aggregation (RSA; Venkatraman et al., 2025), and RSA accuracy actually declines with larger candidate pools.
• Compute reduces variance, not residual bias. For all compared methods, variance generally decreases as inference compute increases, while bias decreases initially and then approaches a plateau. Rankings become more stable, but some pairs remain consistently misranked.
• Scorpio improves verification scaling. Against pointwise scoring and V1's tournament-style pairwise scoring, Scorpio with K = 4 improves peak accuracy by 2–7 percentage points on HMMT and Reasoning Gym games and matches or exceeds the strongest baseline's peak accuracy with 17–41% lower inference cost; on LCBv6 with GPT-OSS-20B it matches the strongest baseline's peak accuracy with 18% lower inference cost. The abstract and introduction state gains of up to 7 percentage points and 17–42% lower cost.
• A 14.3 percentage point gap remains. Across the six verification settings in the main scaling figure, Scorpio's selection accuracy still plateaus an average of 14.3 percentage points below pass@128.
• Random grouping lowers variance faster than tournament pairing. V1, which pairs candidates with similar scores, consistently shows higher variance than Scorpio, likely because such comparisons are harder and yield less consistent rankings; on LCBv6, V1 attains lower bias than Scorpio. Comparing V1 with Scorpio at K = 2 shows similar trends, so Scorpio's faster variance reduction is not solely due to larger group size.
• Group size trades variance against bias. Varying K over {1, 2, 4, 8} with random grouping, pointwise scoring (K = 1) has higher bias than joint evaluation, and larger groups reduce variance faster by allowing more evaluations per unit cost. On LCBv6, however, K = 8 raises bias, potentially because comparing more candidates in one context is harder; K = 4 is the default.
• Residual bias persists at frontier scale. With Gemini-3.1-Pro on Terminal-Bench-2.1, generating N = 8 responses per task and selecting with Scorpio (K = 4), selection accuracy plateaus about 9 percentage points below pass@8; bias plateaus while variance continues to fall (averaged over 8 seeds).
• Tree search improves on code. Using Scorpio to score partial solutions inside DeepConf's search procedure, with evaluations every 2,048 tokens, a retain fraction of ρ = 0.75, K = 4, and M = 3, Scorpio exceeds the strongest baseline's peak accuracy by 10 percentage points on LCBv6. On AIME 2024, where pass@1 is already high, DeepConf and Scorpio reach similar peak accuracy. Increasing the number of scoring repeats from M ∈ {1, 2, 3} improves selected-answer accuracy by 2–6 percentage points across three datasets and two models.
• Aggregation failure is partly bad-mode amplification. Scaling RSA from N = 8 to N = 128 over L = 1 → 10 rounds lowers final accuracy by 6 percentage points despite roughly 16× more compute. On LCBv6 problem #120, responses containing unsupported tool calls grow from 5.8% of the initial pool to 40% of the final pool at N = 128; Scorpio-based pruning of the top N/2 candidates reduces these to nearly zero and raises final accuracy from 40% to 87%.
• Cross-model judges may reduce bias. A preliminary experiment replacing Qwen3.5-4B's self-judgments with judgments from other models reduces ranking bias, suggesting judges whose biases differ from the generator's are a promising direction.

Methodology in Plain English

The authors treat self-evaluation as a ranking problem. Given a pool of sampled solutions, they check every correct–incorrect pair and record whether the model gave the incorrect one a higher score. The fraction of such misordered pairs is the ranking error. Because scoring is stochastic (temperature sampling, and randomized pairings that change a candidate's opponent), they repeat the scoring R = 20 times, estimate each pair's misranking probability, and use unbiased estimators to split the average error into a squared-bias part and a variance part. They then plot these two quantities against inference compute, measured in dollars of input and output tokens, so that different algorithms can be compared at equal cost.

Their proposed scorer, Scorpio, works as follows: shuffle the N candidates into a fresh random order, cut the order into groups of size K, and ask the model to score all K candidates in each group together in a single prompt. Repeat this M times with independent partitions and average each candidate's scores across rounds. The highest average score wins. Because each call handles K candidates, more scoring rounds fit within a fixed budget than with one-candidate-at-a-time scoring, and because groups are random rather than built from similar candidates, repeated judgments are more consistent.

The experiments cover verification (selecting from N = 128 candidates, with the budget capped at the dollar cost of three rounds of pointwise scoring), tree search (scoring and pruning partial solutions during generation, with initial pool sizes N ∈ {8, 32, 128}), and aggregation (scoring and keeping the top N/2 candidates between RSA rounds, with L = 1 → 10 and N ∈ {8, 32, 128}). Models are Qwen3.5-4B in instruct mode and GPT-OSS-20B at medium reasoning effort, with a frontier check using Gemini-3.1-Pro. Benchmarks are HMMT Feb 2025, the 131-question LCBv6 subset of LiveCodeBench (questions released February to May 2025), the 100-problem Reasoning Gym split from Venkatraman et al. (2025), AIME 2024, and Terminal-Bench-2.1. Costs use prices of $0.03 per million input tokens and $0.10 per million output tokens for GPT-OSS-20B, and $0.20 per million tokens for both input and output for Qwen3.5-4B, obtained from pricepertoken.com as of September 13, 2026.

Why This Matters

The paper reframes a practical frustration — throwing more samples and more judging compute at a problem stops helping — as a measurable statistical property of the judge. It gives practitioners a diagnostic (bias versus variance) and a cheap, training-free intervention, and it shows that gains transfer from simple selection to search and aggregation, where scoring errors compound across steps. It also warns that more candidates are not automatically better: RSA-style aggregation can amplify rare bad behaviors, a concrete failure mode with a concrete fix.

Real-world applications:

• Coding assistants and agentic software tools that generate many candidate programs and must pick one, including preventing the spread of malformed or unsupported responses across iterative refinement rounds.
• Mathematical and scientific reasoning assistants that sample multiple solution attempts and need a reliable way to choose among them without an external verifier.
• Search-guided generation systems where budget must be allocated between exploring partial solutions and evaluating them, such as long-horizon agents that must decide which trajectories to continue.
• Evaluation and model-selection pipelines where LLM judges rank outputs, and where understanding judge bias matters for reliability.

Industry relevance comes from the cost framing: Scorpio matches or exceeds baseline peak accuracy at 17–42% lower inference cost in verification, which directly affects serving economics for any product that samples multiple completions and reranks them. The residual-bias result also sets expectations for teams building self-verification into production, since it implies that scaling judge compute alone will not close the accuracy gap.

Future Directions

• Reducing residual ranking bias directly. The authors identify bias reduction as the key step toward closing the selection-to-pass@N gap, and note that both Scorpio and all baselines leave this bias in place at high budgets.
• Heterogeneous or cross-model judges. A preliminary experiment shows that replacing Qwen3.5-4B's self-judgments with other models' judgments can reduce ranking bias; the paper suggests judges whose biases differ from the generator's as a promising direction but does not establish how far this scales.
• Better bias–variance tradeoffs in grouping design. The finding that larger groups (K = 8) can raise bias on LCBv6 while lowering variance indicates that the optimal group size and partitioning scheme are not settled; the paper treats K = 4 as a default balance rather than a principled optimum.
• Applying the decomposition to other test-time algorithms. The authors frame the analysis as a general lens on compute and design choices and encourage future methods to target bias, variance, or both; the exact behavior of bias at budgets beyond those evaluated is not reported.

Target Audience

Researchers and engineers working on LLM reasoning, inference-time scaling, and LLM-as-judge systems who want a principled account of why self-verification saturates, plus a low-cost method they can drop into existing sampling, search, or aggregation pipelines. It is also useful for evaluation practitioners interested in the reliability limits of model self-judgment, and for readers of the ranking and bias–variance literature looking for a new application domain.

Authors’ abstract

Self-verification and aggregation have emerged as popular approaches for bridging the gap between pass@1 and pass@N. However, increasing the number of candidates does not consistently narrow the gap between final-answer accuracy and pass@N. To understand this limitation, we use ranking error to measure how reliably models distinguish their correct generations from incorrect ones. We show that ranking error admits a bias-variance decomposition, with both components shaped by algorithmic design choices and inference compute. Across our experiments, additional compute reduces variance, while bias initially decreases before approaching a plateau. Guided by this analysis, we introduce Scorpio, a simple and general method that jointly scores randomly grouped candidates. Across math, code, and puzzle benchmarks, Scorpio improves verification, tree search, and aggregation. For verification, Scorpio improves accuracy over the strongest baseline by up to 7 percentage points while matching or exceeding that baseline's peak accuracy with 17-42% lower inference cost. For tree search, using Scorpio to score partial solutions and select the final answer improves accuracy by up to 10 percentage points over prior methods. Finally, Scorpio improves the scaling of aggregation-based methods by 11 percentage points on coding tasks.