Calibration-Free Early Stopping for Chain-of-Thought Reasoning via E-processes

AI summary

Overview

Research area: Natural Language Processing / efficient inference for reasoning large language models (test-time compute allocation, sequential statistics). Technical level: Intermediate. The paper relies on anytime-valid sequential inference (e-processes, Ville's inequality), but its core idea can be grasped without measure-theoretic background. Scope: A calibration-free early-stopping rule for chain-of-thought generation that runs two e-processes (commit and abstain) on a shared per-chunk signal and stops when accumulated evidence crosses 1/α, with no held-out calibration data.

What This Paper Is About

Reasoning LLMs produce long chains of thought, and while more tokens often help, many instances keep generating after the answer has stabilized, while others burn the full budget without progress. Existing adaptive-stopping methods halt when a confidence or stability signal crosses a threshold, but those thresholds rarely transfer across models and datasets, and conformal-style calibration (as in Conformal Thinking, CT) only holds if the held-out validation distribution matches deployment. The authors reframe stopping as sequential evidence accumulation along the reasoning trajectory, so that a single user-chosen significance level α provides an anytime-valid alarm bound without any calibration set.

Key Contributions

1. CES framework. A sequential-evidence framework for adaptive chain-of-thought stopping in which two e-processes accumulate wealth along the trajectory, giving an anytime-valid, signal-level false-alarm bound via Ville's inequality with one user-controlled parameter α and no held-out calibration data.
2. Dual e-process construction with position–velocity decomposition. On a single shared signal s_t, the commit stream uses the clipped deviation from a slow EMA (position), the abstain stream uses the sign of the signal's velocity; the two are anti-correlated by construction and handled symmetrically under one decision rule τ = min{τ⁺, τ⁻, T}.
3. Empirical comparison to calibration-based stopping. Across 12 model–dataset combinations, CES matches or exceeds Conformal Thinking's token–accuracy Pareto front using zero calibration data where CT uses n_val = 50 validation trajectories per split.
4. Cross-cell transfer. The same α applies unchanged to new model–dataset cells, including regimes where CT's calibration is infeasible at strict risk targets under the modest validation budget or fails to transfer.

Main Findings

• Best accuracy at the tightest budgets, without calibration. In the dual-stopping setting at token use ≤ 0.2, Ours-Stab reaches 0.337 accuracy versus CT-Conf 0.214, CT-EAT 0.286, and CT-Stab 0.302 (averaged over 40 splits and 12 cells, CT using n_val = 50 and CES using zero calibration data). Upper-only at ≤ 0.2: Ours-Stab 0.317 versus CT-Conf 0.275 and CT-Stab 0.310.
• The advantage is consistent, not a single lucky operating point. At ≤ 0.8 dual stopping, Ours-Stab is 0.677 versus CT-Conf 0.562; at ≤ 1.0, 0.726 versus 0.705. In upper-only at ≤ 1.0, Ours-Stab is 0.726 versus CT-EAT 0.683 and CT-Stab 0.641.
• Signal choice interacts with framework choice. Plugging CT's own signals (confidence, EAT) into the e-process underperforms the stability signal, especially at tight budgets (dual, ≤ 0.2: Ours-Conf 0.131, Ours-EAT 0.158). The authors attribute this to EAT and confidence plateauing near the top of [0, 1] once a chain is on track, so deviations from the slow EMA are near zero and wealth barely accumulates; the stability signal keeps emitting informative increments late in the chain.
• Cross-cell transfer holds without re-calibration. With Qwen3-8B/AIME fixed as the calibration source, the transferred CT threshold is systematically miscalibrated on target cells — calibration is often infeasible in the low-risk regime under n_val = 50, and over-conservative at higher risks — whereas the same α applies identically to any target cell because decisions depend only on the current trajectory's wealth.
• α is a monotone knob on both error types. As α is swept across all 12 combinations, the realized commit false-positive rate and abstain false-negative rate both decrease monotonically, confirming calibration-free control of the two error types.
• Abstain is selective on unsolvable problems. In the Qwen3-8B / MATH-500 outcome analysis, early commit dominates and most commits on solvable problems are correct; unsolvable problems abstain at a substantially higher rate than solvable ones, and early commit captures a non-trivial fraction of correct predictions on unsolvable problems that a full-run policy would lose to overthinking.
• Naive truncation is dominated. A baseline that stops at a fixed fraction of the token budget without any signal is consistently dominated, indicating the gains come from evidence-driven stopping rather than budget cutting.
• Design ablations. For the upper increment φ⁺, sign(Δs) saturates in the high-budget regime, window means (W = 5, W = 15) show near-zero α⁺ sensitivity, and EMA 0.05 decreases monotonically across the full α⁺ range. For the betting fraction, fixed λ = 0.05 and 0.15 under-commit, λ = 0.50 performs comparably to aGRAPA but needs a grid search, and ONS-m collapses λ to zero within the first few chunks of each trace; aGRAPA matches the best fixed λ without hyperparameter search and is the default.

Methodology in Plain English

The method treats a chain of thought as a sequence of chunks and, at every chunk boundary, computes a single scalar signal (confidence, EAT, or answer stability) that has already been smoothed with an exponential moving average at α = 0.3. Instead of asking "is this signal above a threshold someone tuned on a validation set?", it asks "has this trajectory accumulated enough evidence that stopping is justified?"

Two parallel "wealth" counters start at 1 and are multiplied at each chunk by a factor 1 + λ·φ. The commit counter uses φ⁺ = the clipped gap between the signal and a slow-moving average of itself (γ = 0.05), so it grows when the signal runs above its own local trend without any absolute threshold on the signal's scale. The abstain counter uses φ⁻ = the sign of the signal's change, so it grows when the signal is falling — that is, when the trajectory is losing stability. Because one rises exactly when the other falls, they rarely fire together.

The betting fraction λ is set by aGRAPA, clipped to [0, 1/2], which keeps 1 + λφ ≥ 1/2 and makes the wealth process a valid nonnegative supermartingale under the null that the per-chunk increment has nonpositive conditional mean. Ville's inequality then bounds the chance that wealth ever crosses 1/α by α, uniformly over the whole trajectory, so the counter can be inspected at every chunk without inflating false alarms. Stopping happens at τ = min{τ⁺, τ⁻, T}: commit if the commit stream crosses first, abstain to a fallback if the abstain stream crosses first, and use the full-budget answer otherwise.

Experiments use three models (Qwen3-8B, Qwen3-30B-A3B-Instruct-2507, DeepSeek-R1-Distill-Qwen-32B) on four math benchmarks (AIME, MATH-500, OlympiadBench, DeepScaler), following CT's protocol of 40 random splits, n_val = 50, and test capped at 800. Token use accounts for the reasoning prefix plus every forcing-suffix prefill and intermediate-answer decode up to the stopping boundary.

Why This Matters

Impact on research. The paper shifts adaptive reasoning from "choose a threshold" to "accumulate evidence," which decouples the stopping guarantee from a calibration distribution. It also shows that the anytime-validity of e-processes — previously used in other sequential-testing settings — fits the trajectory structure of chain-of-thought, where the decision is whether a rule ever fires along a dependent sequence rather than whether one held-out example crosses a line. The signal-versus-framework interaction (stability favoring the e-process, confidence/EAT favoring CT's sigmoid threshold) is a useful negative result about how much of a system's gain comes from the signal versus the decision rule.

Real-world applications:

• Serving reasoning models under fixed compute budgets, where per-tenant or per-prompt-format calibration data is unavailable.
• Deploying a new model checkpoint or benchmark without rerunning a calibration sweep for every combination of model, dataset, and token budget.
• Routing hard instances to a fallback (larger model, tool use, or human review) when the abstain stream fires.
• Cost-controlled batch inference on math, science, and symbolic tasks where most problems stabilize early.

Industry relevance. The rule needs no validation split in the deployment loop, and the same α carries across cells — attractive for providers who cannot reserve 50 labeled trajectories per model–dataset cell, especially at strict risk targets where calibration is infeasible under a modest budget.

Future Directions

• Composing the signal-level e-process with a downstream conformal layer to obtain a task-level coverage guarantee on top of CES.
• Setting α instance-wise via test-time learning rather than as a fixed global significance level.
• Extending the framework to other forms of test-time scaling, such as parallel reasoning chains.
• Improving the per-chunk signal itself, for example by adding an auxiliary objective during post-training so that the increments φ correlate more directly with answer correctness.

Target Audience

Researchers and engineers working on efficient LLM inference, test-time compute allocation, and adaptive chain-of-thought; practitioners who need early-exit policies that transfer across models and datasets without calibration budgets; and readers interested in applications of anytime-valid sequential inference (e-processes, Ville's inequality) to language-model generation. A reader should be comfortable with the basics of hypothesis testing and moving averages; the formal supermartingale arguments are confined to the appendices.

Authors’ abstract

Reasoning large language models (LLMs) benefit from test-time scaling, motivating adaptive policies that spend tokens when they help and stop early otherwise. Existing rules halt when a confidence or stability signal crosses a threshold, but such thresholds rarely transfer across models and datasets, and validation-calibrated rules lose their guarantee under shifts in test-time horizon or signal dynamics. We reframe adaptive stopping as sequential evidence accumulation along the reasoning trajectory rather than threshold calibration on held-out data. Our framework runs two e-processes on a shared per-chunk signal: a commit stream tracks the signal's position relative to its slow trend, and an abstain stream tracks its signed velocity, making the two anti-correlated by construction. Under signal-level nulls on the per-chunk increments, Ville's inequality yields an anytime-valid signal-level false-alarm bound at user-specified significance levels, with no held-out calibration. Across various model--dataset combinations, the framework matches or exceeds calibration-based stopping on the token--accuracy Pareto front using zero calibration data, and the same significance level transfers unchanged across cells where calibration becomes infeasible.