Revisiting Complete Reasoning Traces for Post-Training

AI summary

Overview

Research area: Post-training of large language models, specifically supervised fine-tuning (SFT) on reasoning trajectories, with extensions to reinforcement learning and on-policy distillation.

Technical level: Advanced. The paper assumes familiarity with SFT pipelines, reasoning traces, attention-based analysis, reinforcement learning post-training, and on-policy distillation.

Scope: The paper questions whether full reasoning trajectories are actually necessary for post-training, using pilot experiments, attention analyses, and token-removal studies to argue that much of a trace is redundant and that partial or endpoint-based training can be as or more effective.

What This Paper Is About

LLMs are commonly post-trained on pre-collected reasoning trajectories, and because these traces contain complex, interwoven paths and detours, they tend to be long. The authors ask whether models genuinely benefit from learning these complete traces during post-training such as SFT, since this question has been underexplored. Their goal is to test the value of full trajectories versus partial ones and to understand why long traces may be unnecessary.

Key Contributions

1. A pilot study comparing training on full reasoning trajectories against partial trajectories, finding that full trajectories give only limited benefit while partial trajectories remain effective even under heavy truncation.
2. An analysis of redundancy in reasoning trajectories using two independent approaches: attention-based analyses and controlled token-removal studies.
3. Evidence that intermediate tokens contribute minimally to final reasoning quality, supporting the idea that models can infer missing steps internally when the trajectory endpoints are known.
4. A demonstration that training with endpoints produces consistent changes in reasoning behavior and also benefits post-training methods based on reinforcement learning and on-policy distillation.

Main Findings

• Full trajectories offer limited benefit: Training on complete reasoning traces yields only limited improvement, contrary to the assumption that longer, more complete traces are better.
• Partial trajectories hold up under heavy truncation: Even when trajectories are truncated substantially, partial traces remain effective for post-training.
• Intermediate tokens are largely redundant: Attention-based analyses and controlled token-removal studies both indicate that intermediate tokens contribute minimally to final reasoning quality.
• Models may fill in gaps themselves: The authors suggest that avoiding redundant information lets LLMs infer coherent alternatives by reconstructing missing steps from internal knowledge, given known trajectory endpoints.
• Endpoint training changes reasoning behavior: Training on endpoints leads to consistent changes in how models reason, rather than simply degrading them.
• Beyond SFT: The endpoint-based approach also benefits reinforcement learning post-training and on-policy distillation, indicating the finding is not confined to supervised fine-tuning.
• Details not in the abstract: The abstract provides no quantitative results, dataset descriptions, model sizes, or baselines; those specifics are not available from the abstract alone.

Methodology in Plain English

The researchers began with a pilot study: they compared models post-trained on full reasoning trajectories with models trained on truncated, partial versions of those same trajectories. To understand why partial traces worked so well, they inspected where the model's attention goes within a reasoning trace, and separately ran controlled experiments in which they removed selected tokens to see how final reasoning quality responded. Together, these two lines of evidence pointed to the same conclusion — the middle of a reasoning path carries little of the useful signal. From there, they tested training on trajectory endpoints, checking whether reasoning behavior changed in a consistent way, and extended the approach to other post-training paradigms such as reinforcement learning and on-policy distillation. Code is released at the linked repository.

Why This Matters

The work challenges a default assumption in post-training: that models should learn complete, unfiltered reasoning traces. If intermediate steps are largely redundant, then collecting, storing, and training on long traces may be wasted effort, and a model's own internal knowledge may substitute for the steps that are dropped.

Potential applications (the abstract does not evaluate these directly; they follow from the paper's claims):

• Cheaper creation of reasoning training data, since shorter traces may suffice instead of full long-form trajectories.
• Reduced training and storage costs when post-training on large reasoning corpora.
• Simpler data pipelines for teams that cannot afford to generate or annotate complete reasoning paths.
• Broader applicability of reasoning post-training to settings where only problem-and-answer style data is available.

Industry relevance: Groups building reasoning-capable models spend significant resources generating long reasoning trajectories for SFT, RL, and distillation. A finding that endpoints carry most of the useful signal could shift how those data budgets and pipelines are structured.

Future Directions

• Determining how much truncation is optimal, and where the boundary lies between a helpful partial trajectory and an uninformative one.
• Clarifying the mechanism by which models infer omitted steps from internal knowledge, and when that inference fails.
• Testing whether the endpoint-based approach generalizes across model scales, domains, and reasoning task types.
• Establishing how to select which portions of a trajectory to keep, rather than dropping intermediate tokens uniformly.

Target Audience

Researchers and engineers working on LLM post-training, reasoning capabilities, or data-efficient fine-tuning; practitioners designing SFT, RL, and on-policy distillation pipelines; and anyone studying how much of a reasoning trace actually carries learnable signal.

Authors’ abstract

Large language models (LLMs) are often post-trained on pre-collected reasoning trajectories to improve their reasoning capability. Such trajectories tend to be long due to complex, interwoven paths, which often include detours on the path toward the answer. However, it has been underexplored whether LLMs indeed benefit from learning complete trajectories in post-training, such as supervised fine-tuning (SFT). Starting from our pilot study, we find that full trajectories provide only limited benefit, while partial trajectories are effective even under heavy truncation. We analyze redundancy in reasoning trajectories through attention-based analyses and controlled token-removal studies, both of which show that intermediate tokens contribute minimally to final reasoning quality. This suggests that avoiding redundant information may allow LLMs to internally infer coherent alternatives by inferring missing steps from their internal knowledge, given known trajectory endpoints. Furthermore, we show that training LLMs using endpoints leads to consistent changes in reasoning behavior, and that it also benefits post-training methods based on reinforcement learning or on-policy distillation, highlighting the need to revisit complete reasoning traces. Code is available at https://github.com/naver-ai/revisiting-trace.