Video Reasoning without Training

AI summary

Overview

Research area: Computer vision and multimodal reasoning, specifically training-free inference-time optimization for video reasoning with Large Multimodal Models (LMMs).

Technical level: Advanced. The paper combines entropy-based analysis of token distributions, a trainable value-cache controller, and formal propositions on entropy bounding, alongside large-scale benchmarking.

Scope: The paper proposes V-Reason, an inference-time method that uses output entropy to shape a video LMM's reasoning behavior without reinforcement learning or supervised fine-tuning.

What This Paper Is About

Video reasoning with LMMs currently depends on expensive reinforcement learning (RL) and long chain-of-thought outputs, which drive up computation at both training and inference time. The authors ask whether the reasoning traces that RL appears to "unlock" can instead be surfaced purely at inference time, by studying the entropy of the model's output distribution as a signal of how it explores and converges on an answer. Their goal is to guide a pretrained instruction-tuned model toward the entropy behavior of stronger or RL-trained models, using no supervision, no dataset, and no RL signal.

Key Contributions

1. The authors present what they describe as the first training-free, purely inference-time optimization method for video reasoning, requiring neither supervised fine-tuning (SFT) nor RL. They state that the problem of "video reasoning without training" has not previously been addressed in the literature.

2. They propose V-Reason, which enhances the micro-exploration and micro-exploitation cycles of baseline instruction-tuned models to improve accuracy, and provide simple theoretical results for the method.

3. They show that V-Reason's entropy-based objective induces a lower and delayed entropy peak during macro-exploration and a lower final entropy during macro-exploitation, matching patterns observed in RL-trained reasoning models.

4. Across six video reasoning benchmarks, V-Reason achieves an average improvement of 1.4% over the base model, narrowing the gap to within 0.6% of the RL-trained Video-R1-7B model, with gains across 3B to 32B and up to 72B LMMs, and it is complementary to multiple state-of-the-art decoding methods.

5. V-Reason produces 21.4% fewer tokens on average than the base Qwen2.5-7B-Instruct model and 58.6% fewer tokens on average than the RL-trained Video-R1-7B model, with wall-clock inference time competitive to the base model and up to 37% lower than Video-R1-7B on average.

Main Findings

• Two-phase entropy structure: All models exhibit a pattern of increasing and then decreasing entropy as tokens are generated, which the authors call macro-exploration (uncertain search through solution trajectories) followed by macro-exploitation (gradual certainty about a solution).

• Quality correlates with entropy shape: Larger, more accurate models exhibit both a later entropy peak and a lower final entropy. The authors note that the signatures of better reasoning are delayed peaks, lower final entropy, and richer micro-cycles — not absolute entropy values.

• Micro-cycles matter: All models go through series of micro-exploration and micro-exploitation cycles (small increases and decreases of entropy) during both macro phases. More or longer such cycles are associated with "longer thinking," lower and delayed entropy peaks, and lower final entropy.

• Small models behave differently: Smaller models (e.g., 3B) peak early and converge prematurely to lower entropy, producing confident but incorrect answers.

• RL models match the pattern: The RL-trained Video-R1-7B model has a slightly lower and much later entropy peak than the Qwen2.5-VL-7B-Instruct baseline used to train it, with final entropy very close to the much larger Qwen2.5-VL-72B-Instruct model. Entropy averaging was performed over the MMVU dataset of 625 videos.

• Benchmark gains: Both (at least one) versions of V-Reason improve the baseline for 15/18 (18/18) model/dataset combinations, with gains of at least 0.9 points for 19/36 combinations and as high as 3.9 points.

• Specific benchmark comparisons: For MMVU with 7B-256px models, V-Reason (Lite) reaches 63.4 points versus a 59.5 baseline and 64.2 for Video-R1. At 128px, V-Reason surpasses Video-R1 on MMVU (65.0 vs. 64.0) and matches it on TempCompass (73.3 each), while nearly closing the gap on VideoMMMU (47.4 vs. 48.8).

• Average accuracies: At the 7B scale, V-Reason/V-Reason(Lite) reaches 54.8%/54.6% (256px) and 55.9%/56.0% (128px) average accuracy without MRA, narrowing the gap to Video-R1-7B (56.6% at 128px) to within 0.6%. At the 3B scale, average gains over Qwen2.5-VL fall in the range of +0.9% to +1.3%.

• Frame and resolution sensitivity: For a comparable compute budget (256px/16 frames versus 128px/32 frames), accuracy improves with more frames, and V-Reason shows larger average gains at 32 frames (+1.4%) than at 16 frames (+1.0%).

• Full versus Lite: Both variants surpass the base model; Lite performs comparably or slightly better in several cases (+1.0% at 3B and +1.4% at 7B for 128px), while the Full variant offers an additional +0.2% improvement at 256px with 16 frames.

• Efficiency: V-Reason substantially reduces output token length (58.6% reduction over Video-R1), translating to latency savings of up to 67% (37% on average).

• Scalability across model sizes: On MMVU, V-Reason gives +3.0% for a Qwen2.5-VL-32B model (72.0 vs. 69.0), +0.4% for a 72B model (73.0 vs. 72.6), and +5.1% for a Qwen3-VL-8B model (69.9 vs. 64.8).

• Objective ablations: Compared to the Qwen-2.5-VL-7B baseline of 61.3 on MMVU, min-entropy (Lite) reaches 62.1 (+0.8), max-entropy (Lite) 63.8 (+2.5), a KV-cache controller (Lite) 61.8 (+0.5), updating the last two layers (Lite) 62.2 (+0.9), and V-Reason(Lite) 65.0 (+3.7).

• Complementarity with decoding methods: Combining V-Reason(Lite) with min-p decoding at temperature 0.3/top-p 0.9 yields 63.8 versus 61.8 for min-p alone (+2.0); with top-H at the same settings, 61.1 versus 60.2 (+0.9). At temperature 1.0/top-p 0.9, min-p drops to 55.0 but min-p plus V-Reason(Lite) restores it to 61.3 (+6.3). V-Reason-7B (Lite) achieves the highest score of 65.0, a further +2.8 gain over the best decoding baseline.

• Limits of the approach: The method is most effective when the solution lies within the model's pretrained knowledge space (e.g., the classification tasks studied); for tasks underrepresented in pretraining, such as the regression task VSI-MRA, training-based approaches remain more effective.

• Qualitative behavior: In the example shown, the baseline model initially follows a promising trajectory but diverges onto an incorrect reasoning path, while V-Reason identifies an alternative path at exactly the point where the baseline falters and reaches the correct answer.

Methodology in Plain English

The authors begin by measuring the Shannon entropy of the model's output token distribution at each generation step, averaging this over many videos, and comparing the resulting curves across models of different sizes and training regimes. From these curves they identify macro-exploration (entropy rising), macro-exploitation (entropy falling), and smaller micro-cycles embedded within both phases.

To act on these observations, they add a small trainable controller, called ΔV, to the value cache of the last decoder layer at the video token positions. All other layers stay frozen and text tokens are untouched. The controller starts at zero and is updated at each generated output token after the first; the first token involves KV-cache prefilling across all layers. A normalization step rescales the updated cache to preserve the original magnitude of the value vector, so the controller applies only a directional update and does not destabilize the forward pass.

The update is driven by an Entropy Switching Loss. A coefficient α, which is either +1 or −1, toggles between encouraging entropy

Authors’ abstract

Video reasoning using Large Multimodal Models (LMMs) relies on costly reinforcement learning (RL) and verbose chain-of-thought, resulting in substantial computational overhead during both training and inference. Moreover, the mechanisms that control the thinking process in these reasoning models are very limited. In this paper, we use the entropy of the model's output distribution as a signal to study and guide reasoning behavior. We discover that high-quality models exhibit a characteristic pattern of micro-exploration and micro-exploitation cycles, followed by a later entropy peak (i.e., longer thinking) and a lower final entropy, indicating more deliberate exploration and confident convergence (i.e., avoid excessive randomness while the model is exploring or thinking through an answer). We then use these novel, theoretically-grounded insights to introduce V-Reason (Video-Reason), an inference-time optimization method that adapts the value cache of the LMM through a lightweight, trainable controller. Our proposed controller is guided by an entropy-based objective, to tune the model's behavior directly at inference, without using any RL or supervised fine-tuning. Our experiments show that V-Reason significantly outperforms the base instruction-tuned models on many video reasoning datasets, narrowing the gap with RL models to within 0.6% accuracy on average. We achieve this without any training, while offering efficiency benefits: V-Reason uses 58.6% fewer tokens than the RL model. Project Page https://deepaksridhar.github.io/vreason.github.io/