ThinkFuse: Trajectory-Aware Test-Time Fusion for Small Reasoning Models

AI summary

Overview

Research area: Test-time compute methods for large language model reasoning — specifically, test-time fusion of reasoning trajectories from small reasoning models (SRMs).

Technical level: Intermediate. The paper assumes familiarity with chain-of-thought reasoning, token-level entropy, perplexity (PPL), and test-time scaling, but the core idea (deciding when to intervene in a reasoning trace) is conceptually accessible.

Scope (one sentence): The paper proposes ThinkFuse, a training-free framework that monitors segment-level uncertainty against a model's own recent uncertainty history and selectively splices in auxiliary reasoning segments when the primary model's trajectory appears unstable, evaluated across five benchmarks and six open-weight models in the 1.2B–4B parameter range.

Metadata: arXiv:2610.07803v1 [cs.AI], 06 Oct 2026, CC BY 4.0. Myunghoon Kang and Jungseob Lee (equal contribution), Jaehyung Seo, and Heuiseok Lim (corresponding author); Korea University, Konkuk University, and Human-inspired AI Research. Code at https://github.com/js-lee-AI/ThinkFuse. (Note: the supplied paper text is truncated mid-way through the Ethical Statements section.)

What This Paper Is About

Small reasoning models can produce long chain-of-thought traces, but once their reasoning enters a wrong path they often cannot recover. Existing test-time fusion methods decide when to step in using local signals at a single decoding step, which can mistake a brief uncertainty blip for genuine reasoning instability — triggering too early, too late, or unnecessarily. ThinkFuse's goal is to make that intervention decision trajectory-aware: compare how uncertain the current reasoning segment is against how uncertain this particular model has been recently, and only then borrow an alternative reasoning segment from a second, auxiliary model.

Key Contributions

1. A trajectory-aware fusion framework. ThinkFuse identifies unreliable reasoning segments by combining an Exponentially Weighted Causal Aggregation (EWCA) uncertainty metric over tokens with an adaptive threshold derived from the primary model's own historical uncertainty statistics and cumulative fusion frequency — no training required.

2. A selective triggering plus compatibility-scoring mechanism. The auxiliary model is invoked only when segment uncertainty exceeds the adaptive threshold, and the winning candidate segment (primary vs. auxiliary) is chosen by the primary model's own perplexity under the current context, so borrowed segments must be compatible with the trajectory in progress.

3. Empirical validation across model families and scales. Experiments on mathematical and knowledge-intensive benchmarks show gains consistent across cross-family and same-family model pairs, including settings where the primary model is smaller than the auxiliary.

4. An efficiency analysis. ThinkFuse reaches higher accuracy with fewer fusion actions and fewer generated tokens than comparable fusion baselines, and does so at a TFLOP budget comparable to AdaFuse and lower than Cool-Fusion.

Main Findings

• ThinkFuse improves on its own primary model on every benchmark tested. With Qwen3-4B as primary, all four evaluated pairs improve over the 89.0 / 92.0 / 53.3 / 54.0 / 29.5 standalone baseline on MATH-500, GSM8K, AIME24, GPQA and NQ-Open. The paper reports that all twenty deltas across the four Qwen3-4B pairs are positive, with eleven significant at the 95% level by bootstrap confidence interval.

• The Qwen3-4B × Ministral pair is the strongest configuration, winning four of five benchmarks. Results: MATH-500 90.0 (+1.0), GSM8K 96.9 (+4.9), AIME24 73.3 (+20.0), GPQA 64.6 (+10.6), NQ-Open 39.5 (+10.0), where deltas are relative to standalone Qwen3-4B.

• Existing test-time fusion baselines degrade sharply on the same model pair. With Qwen3-4B × Ministral, Cool-Fusion falls from 89.0 to 31.0 on MATH-500 and from 54.0 to 11.6 on GPQA; AdaFuse underperforms the standalone primary on four of five benchmarks (45.5 on MATH-500, 72.6 on GSM8K, 33.3 on AIME24, 28.8 on GPQA, and 38.5 (+9.0) on NQ-Open).

• Self-consistency is an inconsistent substitute for fusion. With Qwen3-4B, K=3 gives 89.5 (+0.5) / 94.5 (+2.5) / 53.3 (+0.0) / 51.0 (−3.0) / 39.0 (+9.5) and K=5 gives 88.0 (−1.0) / 94.5 (+2.5) / 53.3 (+0.0) / 50.0 (−4.0) / 39.0 (+9.5) — gains on GSM8K and NQ-Open, no gain on AIME24, and regression on GPQA.

• Redundant fusion hurts. The same-model Qwen3-4B × Qwen3-4B configuration degrades on MATH-500 (75.0, −14.0), GSM8K (90.0, −2.0), AIME24 (40.0, −13.3), and GPQA (28.3, −25.7), while gaining on NQ-Open (38.0, +8.5).

• A smaller primary model still benefits. Pairing Qwen3-1.7B with EXAONE-4.0 raises GPQA from 39.4 to 53.5 (+14.1), exceeding both standalone models. On MATH-500 the gain is +2.0 (85.0), on GSM8K +0.5 (90.5), NQ-Open is unchanged at 28.0, and AIME24 differs by a single problem (30.0, −3.3). This comparison is noted as favoring the standalone baseline, because standalone Qwen3-1.7B is evaluated with the Qwen3-recommended top-k of 20 while ThinkFuse uses the shared decoding setting.

• Efficiency favors ThinkFuse on accuracy per TFLOP. On Qwen3-4B × Ministral: Cool-Fusion uses 39.4 s wall-clock, 86 TFLOPs, 21.8 average accuracy (0.25 acc/TFLOPs); AdaFuse 33.9 s, 74 TFLOPs, 43.7 (0.60); ThinkFuse 68.7 s, 77 TFLOPs, 72.9 (0.95). The longer wall-clock time is attributed to CPU–GPU data transfer during PPL scoring rather than additional computation.

• Fusion behavior correlates with quality. Cool-Fusion fuses the most auxiliary tokens yet produces the shortest thinking-stage outputs, which the authors link to premature redirection of the primary trajectory; AdaFuse uses fewer auxiliary tokens but still shortens the thinking stage on AIME24 and GPQA; ThinkFuse uses auxiliary tokens more sparingly than Cool-Fusion while sustaining a longer reasoning process.

• Ablations confirm each component. Soft budget (λ=50) reaches 90.0 / 96.9 versus no budget penalty (λ=0) at 88.5 / 90.5 — worse than standalone. Segment length T=8 is best (90.0 / 96.9); T=4 gives 87.0 / 93.5, T=16 gives 88.0 / 93.5 and T=32 gives 88.5 / 93.5. Primary PPL scoring gives 90.0 / 96.9 versus Average PPL at 80.5 / 92.0. Entropy gives 90.0 / 96.9 versus a margin-based confidence gap at 87.5 / 98.6.

• Non-reasoning auxiliaries do not work well. With Qwen3-4B as primary, Llama-3.2-3B-IT gives 89.5 (+0.5) / 50.5 (−3.5), Gemma-2-2B-it 87.5 (−1.5) / 51.0 (−3.0), Mistral-7B-Inst-v0.3 84.5 (−4.5) / 44.9 (−9.1), Qwen2.5-3B-Inst 88.0 (−1.0) / 52.5 (−1.5), and Qwen3-1.7B (no-think) 89.5 (+0.5) / 50.0 (−4.0), for MATH-500 / GPQA respectively. On GPQA, every non-reasoning auxiliary degrades performance.

• A qualitative case shows mid-trajectory repair. On AIME 2022 II #11, the standalone Qwen3-4B trace grounds coordinate D prematurely and enters what the authors describe as a dead-end quadratic, answering 0. ThinkFuse triggers a fusion and selects an auxiliary segment that keeps D and C symbolic and enforces both angle-bisector distance constraints before solving, producing the correct answer 180.

Methodology in Plain English

The pipeline works as follows:

1. Generate in fixed-length segments. The primary model writes its reasoning in segments of fixed token length T (default 8). The authors deliberately monitor uncertainty over these fixed-length windows rather than using transition words like "wait" or "alternatively" as segmentation cues.

2. Score each token's uncertainty with entropy. For every token, the framework computes normalized Shannon entropy over the top-r log-probability distribution the model returned, giving a per-token uncertainty value.

3. Aggregate tokens into one segment score (EWCA). Token uncertainties are combined with exponentially weighted causal aggregation. Each token's weight grows with its position in the segment and is multiplied by an accumulation factor over all preceding tokens' uncertainties — so late-token spikes and building uncertainty both matter. The segment score is the weighted average.

4. Compare against an adaptive, personalized threshold. Two exponential moving averages track the model's own recent mean uncertainty and recent deviation scale. The fusion threshold is the recent mean plus a multiple of the recent deviation, multiplied by a penalty that rises quadratically with the running fusion ratio R (the soft budget). As more fusions accumulate, the threshold rises, discouraging further intervention.

5. Fuse only on a spike. If the segment score is below the threshold, the primary model's segment is kept. If the score meets or exceeds the threshold, the auxiliary model generates an alternative segment from the same context, and the primary model's perplexity scores both candidates; the lower-perplexity segment is appended to the context, and generation continues.

6. Evaluate. Six open-weight models — Qwen3-4B, Qwen3-1.7B, DeepSeek-R1-Distill-Qwen-1.5B, Ministral-3B-R, EXAONE-Deep-2.4B, EXAONE-4.0-1.2B — are tested on MATH-500, GSM8K, AIME 2024, GPQA-Diamond, and NQ-Open, against standalone models, self-consistency (K=3, 5), Cool-Fusion, and AdaFuse. Implementation uses vLLM on two NVIDIA A100-80GB GPUs, a shared 16,384-token generation budget, LM Evaluation Harness for GPQA and NQ-Open, and Qwen2.5-Math protocols for MATH-500, GSM8K, and AIME-24. Responses without a parseable final answer within budget are scored as incorrect.

Why This Matters

Impact on research. The paper's distinguishing claim is that prior test-time methods optimized what to fuse (answer, word, span, segment, step) while leaving when to fuse underexplored. Table 1 positions ThinkFuse as the only method combining selective triggering, trajectory-aware calibration, and multi-model fusion — Cool-Fusion fuses at every segment, AdaFuse relies on local token confidence, MUR tracks single-model momentum uncertainty, and SpecReason verifies every drafted step. The result reframes a local-signal problem as a personalization-over-time problem: the same uncertainty value means different things depending on the model's own recent history. It also provides negative evidence that larger auxiliary models help: Mistral-7B-Inst-v0.3 degrades results where much smaller reasoning-specialized auxiliaries help.

Potential real-world applications (implied by the design and limitations rather than demonstrated in the paper):

• On-device or on-premise reasoning assistants where only small models (1.2B–4B) fit in memory.
• Cost-sensitive question answering over technical or scientific material, where a small auxiliary model is cheaper than escalating every query to a large proprietary model.
• Mathematical tutoring or step-checking tools that need to detect and repair a wrong intermediate step rather than only return a final answer.
• Agentic pipelines that must decide mid-run whether to escalate or correct, while keeping latency and token spend bounded.

Industry relevance. The framework is training-free and reuses the current KV cache during PPL scoring to avoid a full extra forward pass over the whole context, which matters for teams that cannot fine-tune models. The efficiency table gives practitioners a concrete trade-off: higher wall-clock time and slightly higher TFLOPs than AdaFuse (77 vs. 74) in exchange for far higher accuracy (72.9 vs. 43.7 averaged over five benchmarks). The reported requirement for reasoning-specialized auxiliaries is a practical deployment constraint — non-reasoning instruction-tuned auxiliaries degraded GPQA across the board in the tested settings.

Future Directions

1. Scaling and generalization of evaluation. The study covers open-weight models in the 1.2B–4B range; the authors state that larger models, closed models, and broader model families need to be tested.

2. Modalities and task types beyond math/science/knowledge. Code generation, multilingual reasoning, and long-context tasks are explicitly named as out of scope.

3. Robustness to weak or unstable primary models. If the primary model emits highly unstable or repetitive segments, uncertainty-based triggering may fail to find useful intervention points; the authors suggest alternative decoding policies or stronger primary-model selection criteria.

4. Reducing dependence on auxiliary complementarity. When the auxiliary shares the primary's failure modes or lacks task-relevant strengths, fusion may not help. Proposed remedies include task-specific auxiliary selection or model-pool routing.

5. Engineering for broader deployment. Segment-level intervention requires identifying the explicit reasoning span and aligning text across different tokenizers; the current implementation uses <think> tag normalization and shared-prefix text alignment, and the authors note more engineering is needed for other chat templates and tokenizer combinations.

6. Hyperparameter search. Segment length and the PPL scoring rule were fixed in the main experiments, with only representative alternatives in ablations; broader searches over threshold and budget hyperparameters might yield further gains.

Target Audience

Researchers working on test-time compute, reasoning-model efficiency, and uncertainty quantification in LLMs; engineers deploying small reasoning models under memory or cost constraints who need a training-free way to improve reasoning reliability; and practitioners interested in multi-model collaboration or routing strategies. Readers focused on training-time methods, large-scale model pretraining, or the specific benchmarks used here (MATH-500, GSM8K, AIME 2024, GPQA-Diamond, NQ-Open) will find the results familiar in format, but the triggering-policy framing and the negative results on redundant and non-reasoning auxiliaries are the most transferable parts.

Authors’ abstract

Small reasoning models (SRMs) have shown strong performance on complex reasoning tasks by generating extended chain-of-thought trajectories, but they often fail to recover once their reasoning enters an erroneous path. Existing test-time fusion methods rely on local fusion signals to determine when to trigger fusion, which can be misled by transient uncertainty fluctuations and may reinforce unstable reasoning trajectories. We propose ThinkFuse, a training-free test-time fusion framework that selectively intervenes in unreliable reasoning segments. ThinkFuse compares segment-level uncertainty shifts with trajectory-level uncertainty trends to identify unstable reasoning points and fuse auxiliary reasoning paths into the primary model's trajectory. Extensive experiments demonstrate that ThinkFuse outperforms baselines on mathematical and knowledge-intensive reasoning benchmarks, with consistent gains across model-family combinations, and remains robust with a smaller primary model. Our analysis shows that ThinkFuse requires fewer fusion triggers and generates fewer tokens, highlighting the efficiency of selective triggering. Our code is available at https://github.com/js-lee-AI/ThinkFuse.