CGES: Confidence-Guided Early Stopping for Efficient and Accurate Self-Consistency

AI summary

Overview

Research area: Natural Language Processing / LLM inference efficiency and test-time scaling. Technical level: Intermediate — the core idea is intuitive, but the paper includes Bayesian posterior derivations and asymptotic consistency proofs that require some probability background. Scope: A Bayesian framework that uses per-response confidence scores to both aggregate LLM samples and decide when to stop sampling, cutting inference calls roughly in half without losing accuracy.

What This Paper Is About

Self-consistency (SC) — asking an LLM the same question many times and taking the majority answer — is a standard way to boost reasoning accuracy, but it always costs a fixed number of model calls and breaks down when the correct answer is rare across samples. This paper asks whether confidence scores attached to each response can replace raw frequency counts as the evidence for choosing an answer, and whether those same scores can tell the model when it has sampled enough. The result is CGES, a method that stops sampling early once one candidate answer accumulates enough Bayesian posterior mass.

Key Contributions

1. Reframing self-consistency as posterior inference. Instead of counting how often each answer appears, CGES treats each (response, confidence) pair as probabilistic evidence and computes a normalized posterior score over candidate answers.

2. A unified aggregation-and-stopping mechanism. Prior methods separate the aggregation rule (e.g., majority vote) from the stopping heuristic (e.g., answer agreement). CGES uses the same posterior scores for both choosing an answer and deciding when further sampling is unnecessary.

3. Theoretical guarantees under two regimes. Theorem 3.2 proves consistency when confidence scores are perfectly calibrated. Theorem 3.5 proves consistency under realistic, noisy confidence, requiring only a directional drift condition: correct answers must carry higher confidence on average than any given wrong answer. Proposition 3.3 bounds the conditional error probability at the stopping time by 1 − γ.

4. Empirical validation on five reasoning benchmarks. CGES reduces average LLM calls by 58% (16.0 → 6.7) while staying within 0.4 percentage points of self-consistency accuracy, and Pareto-dominates SC and heuristic early-stopping baselines on AIME24, MATH500, and MMLU_Pro.

Main Findings

• Large efficiency gains at matched accuracy. The most efficient deployable variant, CGES-DeepConf (B10), uses 6.71 calls on average versus 16.0 for self-consistency — a 58% reduction — at a cost of 0.4 percentage points in average accuracy. The five probability-based CGES variants stay within 0.2 pp of SC on every benchmark except GPQA.

• CGES recovers minority-but-confident answers. Standard majority voting fails when the correct answer is infrequent. Because CGES weights by confidence rather than frequency, it can select a rare answer that carries stronger evidence, which is exactly the regime the noisy-confidence theorem formalizes.

• The theory predicts where CGES fails. GPQA Diamond is the one benchmark where confidence scores barely separate correct from incorrect responses. There, the drift condition μₖ > 0 is weak or violated, the posterior never concentrates, and CGES neither saves calls nor matches SC accuracy. The authors treat this as confirmation of Theorem 3.5 rather than a flaw of the method.

• Fewer calls translate into real latency savings. Because all adaptive methods (ESC, ASC, DSC, CGES) are inherently sequential while SC can batch its fixed budget in one call, raw call counts overstate CGES's advantage. Even accounting for this, wall-clock time per question drops 40–50% versus SC at batch size 1.

• The reward-model variant is a ceiling, not a drop-in. CGES-RM (using Qwen2.5-Math-PRM-7B) improves accuracy by 0.4 pp on MATH500 with only 2.88 calls and 0.7 pp on GSM8K with 1.25 calls — but accuracy drops outside its training distribution, e.g. −10.6 pp on AIME24.

• CGES beats heuristic stopping baselines. ESC and ASC achieve smaller call reductions while losing roughly 2 pp on GPQA; DSC posts the lowest raw call count but loses 1.9 pp on average, with the gap widening at smaller budgets.

• Confidence signals must carry directional information. The requirement is weaker than calibration or independence — it permits cases where the correct answer is sampled less often than a wrong one, as long as correct emissions score higher confidence on average.

Methodology in Plain English

The researchers start by asking the LLM a question once, and recording both the answer it gives and a confidence score for that answer (derived from token probabilities, a learned scorer, or a reward model). They then treat each candidate answer in turn as a hypothesis: "assume this is the correct answer." Under that hypothesis, an observation that matches the candidate contributes its confidence score as evidence in favor; an observation that contradicts it contributes a small fraction of the remaining probability mass. Multiplying these contributions across all samples yields an unnormalized score per candidate, which is then normalized into a posterior.

The algorithm samples one response per question, computes posteriors, and repeatedly queries only the questions whose top posterior is still below a threshold γ. Questions drop out of the sampling loop as soon as their leading candidate exceeds γ, and the process halts when all questions are resolved or a budget B is exhausted. Crucially, γ controls compute rather than abstention — if nothing crosses the threshold, CGES still returns the current argmax.

To validate this, the team ran experiments on AIME24, MATH500, GSM8K, MMLU_Pro, and GPQA Diamond using DeepSeek-R1-Distill-Qwen-7B for the harder tasks and Qwen2.5-7B for the rest. They used five confidence signals (LNS, DeepConf with two variants, MARS, and a process reward model) and averaged results over 10 seeds. For the main table they report the "efficient" threshold: the smallest γ within 0.2 pp of SC accuracy. Pareto curves sweep γ across [0.7, 0.9999].

Why This Matters

Impact on research. The paper's central move is architectural: it collapses two separately-studied problems — how to aggregate samples and when to stop sampling — into one probabilistic object. This gives test-time scaling a principled stopping criterion with a formal error bound, rather than a heuristic threshold. The noisy-confidence theorem is arguably more important than the ideal-case result, because it shows the framework survives the miscalibration that real LLM confidence scores exhibit. It also connects confidence estimation research to inference-efficiency research, two literatures that have largely developed in parallel.

Real-world applications:

• Cost-sensitive LLM deployment. Any production system paying per-token or per-call can use CGES to cut inference spend roughly in half on reasoning-heavy tasks without a measurable accuracy hit.
• Latency-bound interactive systems. Chatbots, coding assistants, and agentic pipelines benefit from the 40–50% wall-clock reduction, since adaptive methods are sequential by nature and users feel each round trip.
• High-stakes verification workflows. Mathematical proof checking, medical question answering, and scientific reasoning tasks where the strongest evidence should override the most common answer are natural fits — CGES's minority-recovery property is precisely what's needed when a rare correct answer is drowned out by confident-sounding errors.
• Resource-constrained edge inference. Reducing the number of required model calls makes multi-sample self-consistency viable on hardware where the 16-call budget for standard SC is prohibitive.

Industry relevance. The method is training-free and model-agnostic, which lowers adoption barriers considerably. It requires only a response-level confidence signal, and the paper explicitly notes that uncertainty quantification methods can serve as proxies when dedicated confidence estimators are unavailable. The companion release of code at a public GitHub repository makes reproduction straightforward. For teams already running self-consistency in production, CGES can be swapped in as a wrapper around the existing sampling loop.

Future Directions

• Better confidence estimators. The paper's own failure analysis of GPQA shows that CGES's ceiling is set by signal quality. Developing confidence scores that separate correct from incorrect responses on tasks where current estimators are weak — GPQA being the clear example — is the most direct lever for extending the method's reach.
• Dynamic sample-count prediction. The authors suggest predicting the required number of samples directly from confidence signals, rather than relying on a fixed threshold γ and budget B chosen per benchmark. This would remove the tuning step and adapt automatically to problem difficulty.
• Handling questions with no correct answer among candidates. Remark 3.6 acknowledges that when the true answer never appears in the sampled set, no sampling-based scheme can recover it, and the consistency guarantees do not apply. The authors note that a high γ can flag such cases but leave a no-answer option and formal abstention guarantees to future work.
• Extending stopping rules beyond asymptotic guarantees. The consistency results are asymptotic and explicitly do not preclude premature stopping before the true answer first appears. Connecting the finite-sample stopping time to bounded error probabilities under realistic assumptions remains open — Proposition 3.3 gives an exact bound only in the ideal calibrated case.

Target Audience

This paper is best suited for machine learning researchers and practitioners working on LLM inference efficiency, test-time scaling, and probabilistic aggregation methods. It will be directly useful to engineers building cost- or latency-sensitive LLM systems who already understand self-consistency and want a principled alternative to majority voting. Readers with a background in Bayesian inference will extract the most from the theoretical sections, though the algorithmic intuition is accessible to anyone familiar with majority-vote ensembling. Those specifically interested in confidence estimation, uncertainty quantification, or the theoretical foundations of sampling-based inference will find the noisy-confidence analysis the most novel contribution.

Authors’ abstract

Large language models (LLMs) are often queried multiple times at test time, with predictions aggregated by majority vote. While effective, this self-consistency (Wang et al., 2023) strategy requires a fixed number of calls and fails when the correct answer is infrequent. We introduce Confidence-Guided Early Stopping (CGES), a Bayesian framework that forms posteriors over candidate answers and adaptively halts sampling once one answer accumulates enough posterior mass. We prove guarantees in both an ideal calibrated regime and a realistic noisy-confidence regime under a directional drift condition. Averaged over five reasoning benchmarks, CGES reduces the average number of calls by 58% on average (from 16.0 to 6.7) while matching its accuracy within 0.4 percentage points of self-consistency.