The Zero-Step Thinking: An Empirical Study of Mode Selection as Harder Early Exit in Reasoning Models

AI summary

Overview

Research area: Efficient reasoning in large reasoning models (LRMs) — specifically adaptive mode selection and early exit for chain-of-thought inference.

Technical level: Intermediate. The paper uses light formal notation (notably a unified Exit(·) function) but is primarily an empirical benchmark study, so readers familiar with chain-of-thought prompting can follow it.

Scope: A systematic empirical evaluation of nine baselines on whether Mode Selection (choosing Thinking vs NoThinking before any reasoning happens) can be solved by methods originally designed for Early Exit.

What This Paper Is About

Large reasoning models often "overthink," spending large numbers of tokens on reasoning chains that go beyond what a problem needs, which wastes computation. One fix is Early Exit, which decides during reasoning when to stop; another is Mode Selection, which decides before reasoning starts whether to use a long chain-of-thought (Thinking) or a short one (NoThinking, triggered by inserting fake thoughts such as Okay, I think I have finished thinking.). The paper's core claim is that Mode Selection is a strictly harder variant of Early Exit — it must make the same stop-or-continue judgment with zero real reasoning steps available, using only the input and hand-crafted fake thoughts.

Key Contributions

1. A unified formalization of Mode Selection as a harder Early Exit problem. The authors frame both tasks under the same Exit(·) function: Early Exit iteratively performs Exit(Q, T_{<i}) at each reasoning chunk, while Mode Selection performs a single Exit(Q, T_0^{fake}) at the zero-step, replacing dynamically generated thoughts with pre-defined fake thoughts.

2. A systematic empirical study across nine baselines and four benchmarks. The evaluated methods are FlashThink, PromptConf, Dynasor-CoT (prompt-based), ProbeConf, DEER, Entropy (internal-state-based), plus three newly introduced baselines: Thinking, NoThinking, and Pre-judge. Evaluation spans GSM8K, MATH-500, AIME25, and GPQA-D on DeepSeek-R1-Distill-Qwen at 1.5B, 7B, and 32B.

3. A demonstration that prompt-based methods fail under minimal information, while internal-state methods do better but remain unstable. FlashThink, for example, reaches a 0% NoThinking ratio (NR) in every scenario across all three model sizes.

4. Evidence that standard evaluation metrics are insufficient to explain method behavior. The authors apply ROC-AUC, Expected Calibration Error (ECE), and Brier score, and find that these metrics are informative for the 7B model but less so for the 1.5B model, leaving the underlying causes of performance differences unexplained.

Main Findings

• Mode Selection is harder than Early Exit because of decision timing. Early Exit monitors iteratively and can use previously generated thinking tokens; Mode Selection monitors only once at the start and has no question-specific reasoning information, making its information budget strictly smaller.

• Prompt-based approaches largely fail. With only the fake thoughts T_0^{fake} available, FlashThink judges reasoning necessary in every case, producing a 0% NR rate on GSM8K, MATH-500, AIME25, and GPQA-D for all three model sizes. The authors attribute this to weak classification capability given minimal hand-crafted information.

• PromptConf shows dataset- and scale-dependent behavior. On the 1.5B model it improves AIME25 accuracy by 6.7 points while cutting token usage by 36.0%, but its effectiveness declines as model size grows to 7B and 32B.

• Internal-state methods generally perform better and sometimes beat the full Thinking baseline. On the 1.5B model, DEER and Entropy retain accuracy better on GSM8K and GPQA-D, while ProbeConf gives higher token compression on MATH-500 and a 6.7-point AIME25 gain with 26.6% token compression. On the 32B model, DEER reaches 65.7 accuracy on GPQA-D versus 62.1 for Thinking and 62.6 for NoThinking, while reducing tokens.

• Larger models narrow the gap between methods. At 1.5B, mode-selection methods clearly separate from the random baseline in the accuracy-versus-token trade-off (threshold λ swept from 0.1 to 1.0). At 7B the gap narrows and even the best method, ProbeConf, occasionally underperforms the Random baseline; at 32B the effect is more pronounced.

• The NoThinking advantage reverses at 32B on hard datasets. On MATH-500, NoThinking uses 3,550 tokens versus 3,462 for Thinking, and on AIME25 it uses 13,933 tokens versus 11,155, while accuracy drops from 66.7 to 60.0. The authors hypothesize that LRMs have internalized the reasoning process, so forcing a bypass can cause the model to restart reasoning — a trend they note was also observed in QwQ-32B by prior work.

• ROC-AUC confirms high sensitivity to dataset and model. No single method is best everywhere. ProbeConf attains the highest ROC-AUC of 72.2 on AIME25 with the 1.5B model, but this advantage does not transfer to 7B or to other datasets.

• Calibration metrics align with ROC-AUC only at 7B. On the 7B model, DEER achieves the lowest Brier score of 0.181 on AIME25, matching its best ROC-AUC. On the 1.5B model these metrics are less informative, which the authors attribute to a substantial performance drop under NoThinking mode making both internal and external scores less reliable.

• NoThinking is not uniformly cheaper or worse. On the 32B model it slightly exceeds Thinking on GSM8K (95.9 vs 95.8) and GPQA-D (62.6 vs 62.1), but on the 1.5B model it drops GSM8K from 85.7 to 72.7 while cutting tokens from 2,455 to 261 (a 89% reduction).

Methodology in Plain English

The authors take methods built to decide when to stop chain-of-thought reasoning mid-stream and repurpose them to decide whether to reason at all at the very beginning. They construct a NoThinking prompt template that inserts pre-defined fake thoughts between <think> and </think>, so the model is effectively told the thinking is already finished and only needs to produce a conclusion.

For each dataset and each of the three model sizes, they first sample instances in both Thinking and NoThinking modes at temperature 0.6. They then run each baseline to produce a decision flag for each instance — either from a verification model or prompt (prompt-based) or from hidden states and output logits (internal-state-based) — and route the instance accordingly. Prompt-based baselines that produce discrete scores (PromptConf, Dynasor-CoT) use the highest score as the threshold λ; for DEER, ProbeConf, and Entropy, fixed thresholds do not capture optimal performance, so the authors manually select thresholds, preferring ones with significant accuracy improvements and otherwise matching accuracy or NR across methods.

Evaluation uses four benchmarks: GSM8K (1,319 elementary math problems), MATH-500 (500 competition-level problems, the OpenAI-curated subset), AIME 2025 (30 problems), and GPQA Diamond (198 PhD-level physics, chemistry, and biology questions, where the paper reports domain experts achieve 69.7% accuracy). Decoding is zero-shot with the prompt "Please reason step by step, and put your final answer within \boxed{}", a maximum generation length of 16,384 tokens, and rule-based equivalence checking against numerical or categorical ground truths. Beyond accuracy, token count, and NR, the analysis adds ROC-AUC curves, ECE, and Brier score. Entropy is excluded from the calibration analysis due to numerical issues. Code is released at https://github.com/Trae1ounG/Zero_Step_Thinking.

Why This Matters

Impact on research. The paper reframes adaptive reasoning as an information-availability problem rather than an architecture problem, giving a clean way to compare routing methods against early-exit methods under one abstraction. It also shows that current evaluation metrics (accuracy, tokens, NR) hide instability, and that standard calibration metrics do not explain behavior at small scale — a signal that new diagnostics are needed. The negative result on FlashThink and the mixed results on prompt-based methods temper the assumption that existing early-exit techniques transfer straightforwardly to mode routing.

Real-world applications.

• Serving systems that route user queries between cheap short-answer paths and expensive long-reasoning paths to control latency and cost.
• Agentic pipelines where each subtask must decide whether it warrants deep deliberation or a direct answer.
• Cost budgeting for high-volume math and science tutoring or homework-help products, where per-query token spend dominates economics.
• Hybrid inference modes shipped in production models (the paper cites Qwen3 and DeepSeek-V3.1 as adopting such hybrid thinking modes).

Industry relevance. Token cost and latency are direct operating expenses for any deployed reasoning model. The finding that a well-tuned internal-state monitor can match or exceed full Thinking accuracy at lower token counts on the 32B model suggests production-worthy savings, while the instability across datasets and scales warns against shipping a single fixed threshold. The reversal at 32B — where NoThinking can be more expensive on hard math — is an operational hazard for anyone assuming short mode is always cheaper.

Funding. The work was supported by the National Key R&D Program of China (No. 2022ZD0160503), the Beijing Natural Science Foundation (L243006), and the National Natural Science Foundation of China (No.62376270).

Future Directions

1. Investigate why large models restart reasoning under NoThinking. The 32B model generating more tokens in NoThinking than Thinking on MATH-500 and AIME25 is unexplained by the paper; understanding the internalized-reasoning mechanism could yield better fake-thought templates.

2. Develop metrics that explain, not just rank, method behavior. ROC-AUC, ECE, and Brier score fail to account for the 1.5B results, and the authors explicitly call for deeper study of the internal mechanisms of Thinking and NoThinking.

3. Build more stable mode-selection methods that exploit internal states. Internal-state approaches beat prompt-based ones in most scenarios but suffer instability across datasets and scales, so robustness — not just peak accuracy — is the open target.

4. Reconsider threshold selection. Optimal λ varies unpredictably across tasks and models, and the authors had to manually pick thresholds for DEER, ProbeConf, and Entropy to show their best results; automated or adaptive thresholding remains unaddressed.

Target Audience

Researchers and engineers working on efficient inference, adaptive reasoning, and chain-of-thought control in large reasoning models. It is most useful to readers already familiar with reasoning-model prompting (thinking delimiters, fake thoughts, NoThinking mode) who want an evidence-based picture of which routing signals work — and where they break — rather than a new method. Practitioners designing cost-aware inference routing will find the benchmark tables and the 32B reversal directly actionable; readers looking for a proposed new algorithm will not find one, since the paper is explicitly an empirical study.

Authors’ abstract

Reasoning models have demonstrated exceptional performance in tasks such as mathematics and logical reasoning, primarily due to their ability to engage in step-by-step thinking during the reasoning process. However, this often leads to overthinking, resulting in unnecessary computational overhead. To address this issue, Mode Selection aims to automatically decide between Long-CoT (Chain-of-Thought) or Short-CoT by utilizing either a Thinking or NoThinking mode. Simultaneously, Early Exit determines the optimal stopping point during the iterative reasoning process. Both methods seek to reduce the computational burden. In this paper, we first identify Mode Selection as a more challenging variant of the Early Exit problem, as they share similar objectives but differ in decision timing. While Early Exit focuses on determining the best stopping point for concise reasoning at inference time, Mode Selection must make this decision at the beginning of the reasoning process, relying on pre-defined fake thoughts without engaging in an explicit reasoning process, referred to as zero-step thinking. Through empirical studies on nine baselines, we observe that prompt-based approaches often fail due to their limited classification capabilities when provided with minimal hand-crafted information. In contrast, approaches that leverage internal information generally perform better across most scenarios but still exhibit issues with stability. Our findings indicate that existing methods relying solely on the information provided by models are insufficient for effectively addressing Mode Selection in scenarios with limited information, highlighting the ongoing challenges of this task. Our code is available at https://github.com/Trae1ounG/Zero_Step_Thinking.