DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning

AI summary

Overview

• Research area: Natural language processing, especially inference-time control of reasoning models.
• Technical level: Intermediate.
• Scope: The paper introduces DoubtLess, a training-free decoding method that uses a smaller model to help a larger reasoning model avoid repetitive checking while preserving useful reasoning.

What This Paper Is About

Reasoning models can continue thinking after they have already reached an answer, repeating checks that add tokens without adding much information. Existing ways to shorten this reasoning can reduce accuracy or require training the model. DoubtLess aims to reduce redundant reasoning at decoding time, without changing the large model’s weights.

Key Contributions

1. A contrastive decoding method: A small model reads the large model’s current reasoning under a brief instruction and a thorough instruction. The difference between its predictions steers the large model toward the brief thinker’s next-token preferences.
2. A broad evaluation against inference-time baselines: The authors compare DoubtLess with five other methods across seven benchmarks covering competition mathematics, graduate-level science, and programming.
3. Evidence that the method targets later checking: The analysis finds that most of the removed tokens occur after the correct answer first appears in the reasoning, rather than during the initial attempt to solve the problem.
4. Extensions beyond the main evaluation: The authors test DoubtLess in a self-debugging coding agent, on other model families, and in combination with a model already trained to produce shorter reasoning.

Main Findings

• Less reasoning with reported unchanged accuracy: With Qwen3-32B steered by Qwen3-0.6B, DoubtLess removes 31% of thinking tokens across the seven benchmarks. The paper reports unchanged accuracy and says DoubtLess removes more tokens overall than the five inference-time baselines.
• Savings mostly come after an answer appears: Across the benchmarks where the answer could be located, 65–89% of the tokens removed by DoubtLess came after the correct answer first appeared in the chain of thought. The two small-model thinkers’ next-token distributions diverged more after the answer than before it.
• The method does not simply ban hesitation words: Words such as “Wait” and “But” remain in the reasoning. DoubtLess instead makes another round of checking less likely when an answer is already present.
• Results extend to other model families: On AIME 2025 and AMC 2023, DoubtLess removes 25% and 40% of thinking tokens, respectively, in tests of EXAONE-4.0 and Granite-4.2. The paper reports that these reductions exceed those from a concise prompt.
• It also reduces coding-agent reasoning: On HumanEval+, the self-debugging agent uses 20% fewer thinking tokens per turn, with its pass rate improving by 2.0 points over plain decoding.
• It can complement training-based compression: When applied to a Qwen3-32B model fine-tuned with LACONIC, DoubtLess further reduces thinking tokens by 29–52% across the reported mathematics, science, and coding-agent evaluations, with unchanged task performance.
• Token savings can also reduce latency: In a test on a single B200 GPU, DoubtLess averages 119.4 seconds per problem, compared with 156.2 seconds for plain decoding and 137.5 seconds for a concise prompt. The authors report latency reductions of 24% and 13%, respectively.

Methodology in Plain English

At each step while the large model is writing its chain of thought, the smaller model reads the same problem and reasoning prefix twice. One instruction asks it to go straight to the answer without repeating checks; the other asks it to double-check thoroughly. The difference between their next-token preferences is scaled by a strength and added to the large model’s own token scores.

This adjustment is applied only to tokens the large model already considers among its top candidates. It operates during the chain of thought, while the final answer is generated by the large model alone. The large model’s own stopping-token scores are preserved, so the small model influences which reasoning continues but does not directly decide when the chain of thought ends. The default strength in the main experiments is α = 5.

The experiments use Qwen3-32B with Qwen3-0.6B as the main model pair, and compare it with plain decoding and five inference-time baselines. The evaluation covers AIME 2024, AIME 2025, HMMT February 2025, AMC 2023, GPQA-Diamond, OlympiadBench, and LiveCodeBench. The authors also test other model families, a self-debugging coding agent, and a model trained for shorter reasoning.

Why This Matters

For research, DoubtLess offers a way to study and control redundant reasoning without retraining the large model. Its analysis also suggests that useful initial reasoning and wasteful repeated checking may be affected differently by decoding interventions.

Potential applications supported by the paper’s evaluations include:

• Competition-math systems: Reduce the length of generated solutions while retaining reported accuracy on mathematics benchmarks.
• Science question answering: Apply the method to graduate-level questions represented by GPQA-Diamond.
• Code generation: Reduce thinking within a self-debugging coding agent that writes programs, tests them, and repairs failures.
• Reasoning systems already trained for brevity: Add decoding-time compression to a model fine-tuned with LACONIC.

For industry, shorter outputs may reduce generation time and token usage, while the method avoids a separate training run for each large model. It does require a smaller model that shares the large model’s tokenizer, and it adds extra inference computation: the paper reports that the two small-model passes add about 4% to the FLOPs of the 32B model.

Future Directions

• Test broader model and task coverage: The reported evaluation focuses on particular reasoning benchmarks, model families, and a coding-agent setup. Further work could assess how well the method transfers to other settings.
• Improve inference efficiency: The small model performs two forward passes at each reasoning step. Reducing that overhead could make the approach more practical at scale.
• Explore adaptive steering strength: The experiments show that stronger steering removes more tokens but can eventually harm accuracy. A method that adjusts the strength to the context or confidence could explore this trade-off.
• Study which checks should be retained: DoubtLess is designed to reduce repeated checking, but the paper’s examples also show that some checks can catch mistakes. Better ways to distinguish valuable verification from repetition remain an open question.

Target Audience

This paper is most useful for NLP and machine-learning researchers working on reasoning models, decoding, or inference-time efficiency. It may also interest teams building math, science, and coding assistants who want shorter reasoning traces without additional training, as well as practitioners studying how to reduce latency and token use while preserving task performance.

Authors’ abstract

Reasoning models achieve strong performance through long chains of thought (CoT), but often spend more tokens than needed to solve a problem. Existing ways to shorten the CoT require additional training for each model, add a fixed steering vector to the model's hidden states, or use decoding heuristics such as stopping early or banning words like "wait", which can reduce accuracy. We propose DoubtLess, a training-free decoding method in which a much smaller reasoning model steers the large one towards brevity. The small model reads the large model's CoT so far twice, once prompted to be brief (the brief thinker) and once prompted to be verbose (the verbose thinker), and the difference between its two logit vectors, scaled by a strength, is added to the large model's logits at every step of the CoT. On seven benchmarks of competition mathematics, graduate-level science and competitive programming, DoubtLess shortens the CoT of Qwen3-32B by 31% with unchanged accuracy, outperforming five inference-time baselines. We also show that DoubtLess reduces thinking tokens per turn by 20% in a self-debugging coding agent with unchanged task performance. It also generalizes to two other model families with unchanged accuracy. When combined with training-based compression, it further shortens the CoT of a model already trained for brevity. Ablations and behavioral analysis show that DoubtLess reduces repeated checking, with most token savings coming after the correct answer first appears in the CoT. Our code is available at https://github.com/leoluolol/DoubtLess.