RECAP: Looking Once Is Not Enough for Vision-Language Reasoning

AI summary

Overview

Research area: Computer Vision / multimodal large language models — specifically reinforcement-learning post-training for vision-language models (VLMs) that produce long chains of thought.

Technical level: Advanced. The paper combines mechanistic attention analysis, GRPO-style policy optimization, and a small amount of reinforcement-learning theory (a separation proposition and an oracle variance theorem). Readers need familiarity with policy-gradient methods and transformer attention to follow the formal sections, though the core intuition is stated plainly.

Scope (one sentence): The paper identifies "visual revisit" — the moment a long-CoT VLM turns its attention back to the image after a stretch of language-only reasoning — as a causal, localized signal of correctness, and builds a GRPO-compatible training method (RECAP) that uses that signal for credit assignment without changing rewards, rollouts, or inference cost.

What This Paper Is About

Long chain-of-thought VLMs often drift away from the image as they generate: after an initial look, they can produce fluent reasoning that is no longer supported by visual evidence. Standard RL post-training (GRPO-style) gives only a final-answer reward, so it can say a trajectory ended correctly but not which intermediate steps should have returned to the image. The paper's goal is to detect those "revisit" moments inside the model's own attention trace and use them to decide where final-answer credit should be concentrated.

Key Contributions

1. Identifies and validates visual revisit as a temporal grounding signal. Using peak-masking, peak-injection, signal-control, and head-localization analyses across three VLM backbones, the authors show revisit is predictive of and causally linked to correctness, and is not explained by raw attention mass, token entropy, logit margin, or hidden-state norm.

2. Proposes RECAP, a GRPO-compatible clipped-surrogate method that converts detached revisit traces into a credit-assignment signal through three components: a trace-augmented estimator input, revisit-conditioned dynamic discounting, and a revisit-gated value predictor. It requires no extra rewards, no additional rollouts, and no inference-time changes.

3. Provides theory and a diagnostic metric. Proposition 1 shows dynamic discounting can induce retroactive credit couplings that same-step local reweighting cannot express; Theorem 1 gives an oracle conditioning analysis of when revisit-gated value prediction reduces return-residual variance, with the trajectory-dependent surrogate bias made explicit in Appendix B. The paper also introduces RAP (Revisit-Aware Propagation), a method-agnostic training diagnostic.

4. Demonstrates broad empirical gains: three 7–8B backbones, a 72B LoRA-tuned model, zero-shot Chinese benchmarks, visual perturbations, a human study, and composition with VPPO.

Main Findings

• Revisit peaks predict correctness. Across Qwen2.5-VL-7B, InternVL3-8B, and LLaVA-OneVision-7B, correct rollouts show recurring cross-modal attention peaks, while failed rollouts lose or mistime them.

• The link is causal, not correlational. Masking revisit peaks lowers HallusionBench accuracy by −4.9 / −4.6 / −4.8 points (mean −4.8), versus −1.3 / −1.0 / −1.2 (mean −1.2) for masking random non-peaks. Injecting revisit-like peaks into failed rollouts gains +3.0 / +2.7 / +3.1 (mean +2.9), while random injection gives −0.9 / −0.4 / −0.6 (mean −0.6).

• Peaks align with genuine decision points. Revisit peaks match annotated decision steps at 42.1% / 60.8% / 61.2% (mean 54.7%) against chance baselines of 17.2% / 15.8% / 19.4% (mean 17.4%).

• The effect is head-localized. The top-10 heads carry more than 40% of revisit mass across backbones, and zeroing them reproduces most of the accuracy drop from masking cross-modal attention entirely (zeroing all CMA costs −7.6 points).

• RECAP beats GRPO consistently on hallucination. Table 1 shows HallusionBench gains of +3.8, +3.5, and +4.1 points over GRPO on Qwen2.5-VL-7B, InternVL3-8B, and LLaVA-OneVision-7B, corresponding to +6.5, +6.1, and +6.5 over the SFT bases, with reasoning and text-only scores largely preserved. On Qwen2.5-VL-72B LoRA, RECAP gains +3.1 over LoRA-GRPO with no inference-time cost.

• Component ablations confirm each piece matters. Removing the dynamic discount causes the largest drop (66.6 → 64.3), followed by removing gated value prediction (65.2, with advantage variance rising to 1.70×), then the trace-augmented estimator state with K=0 (65.8). Removing the stop-gradient on the revisit signal collapses to 60.8 with variance 2.35×.

• Dynamic discounting beats same-signal reweighting. Using the identical signal as a local reweight w(r̃)A gives 64.6 versus 66.6 for the dynamic discount, a gap of +2.0 / +1.6 / +2.1 points on Hallusion across the three backbones — the empirical counterpart of Proposition 1.

• Substituting other signals fails. Replacing revisit with token entropy (63.2), logit margin (63.0), or hidden-state norm (63.5) on Qwen underperforms full revisit at 66.6, with correlations to |A_t| of ρ=0.29, 0.24, and 0.36 respectively.

• Revisit without RL does not recover the gain. SFT + revisit re-injection reaches 62.7 / 61.4 / 59.9, versus 66.6 / 64.7 / 63.4 for RECAP — a +3.9 / +3.3 / +3.5 margin over the best no-RL baseline.

• RECAP composes with VPPO. VPPO + RECAP reaches 68.0 Hallusion (versus 66.6 for RECAP alone, 64.2 for VPPO, 62.8 for GRPO) with the lowest advantage variance at 0.98×.

• Human evaluators prefer it. On 100 HallusionBench/MathVista questions with non-identical CoTs, ten trained annotators give RECAP over 60% of image-groundedness wins against both GRPO and VPPO at Fleiss' κ ≥ 0.5 (McNemar's p<0.01), and an LLM-as-judge agrees with the human majority on 72% of items.

Methodology in Plain English

The authors first measure cross-modal attention — how much each generated text token attends to image tokens — averaged over a fixed set of middle layers and all heads. Absolute attention is a poor grounding score, since some tokens legitimately need the image and others legitimately do not. So they define a revisit as a local rise above a recent baseline: compare the current attention to the average of the previous 16 tokens, keep only positive jumps, and squash the result into a 0–1 value with a temperature-scaled sigmoid.

That number is not used as a reward. Instead, after a rollout is sampled by ordinary GRPO, the detached revisit trace is threaded into three training-time places: the value/advantage estimator sees recent revisit values as extra input; the discount factor is raised at high-revisit positions, so a future revisit can retroactively increase credit for an earlier decision; and two value heads (one "local/language" and one "visual") are mixed by the current revisit value. The final-answer reward, the sampling policy, the clipped ratio, and the KL anchor are all unchanged, and the revisit trace is discarded at inference.

To test the claim, they run a two-stage pipeline: supervised fine-tuning on 50K filtered long-CoT examples, then RL on 30K disjoint verifiable-answer questions (20K for the 72B model) with group size M=8. Headline results use 8 seeds; ablations use 5. They also run interventions (masking and injecting attention at peaks versus random positions) and head-level zeroing to move from correlation to causation.

Why This Matters

Research impact. The paper reframes visual grounding in multimodal reasoning as a temporal problem rather than a static one. Most existing fixes — visual re-injection at inference (TVC), visual-signal reweighting (VPPO), learned anchors (VAPO) — do not ask when the image should re-enter a reasoning chain. RECAP supplies a cheap, reward-free handle on that question, plus a diagnostic (RAP) other methods can be scored with. The head-localization result also connects training-method design to mechanistic interpretability.

Real-world applications:

• Medical and scientific figure analysis, where a model must return to a chart or scan at the exact step where a numeric claim is made, not just at the start.
• Document and diagram QA in enterprise workflows, where fluent but unsupported conclusions are costly.
• Accessibility tools that narrate or answer questions about images and must stay faithful to what is actually depicted.
• Autonomous inspection and safety review, where a reasoning trace that silently stops consulting the image is a failure mode, not a stylistic flaw.

Industry relevance. RECAP plugs into existing verifier-based RL pipelines with no extra reward model, no extra rollouts, and zero inference overhead — the properties that determine whether a post-training trick ships. The gains appear on three widely used open backbones and on a 72B LoRA setting, and they compose with VPPO, so teams already running GRPO-style training can adopt it incrementally.

Future Directions

• Beyond attention as a grounding proxy. The authors note that benefits may shrink for short answers, text-only tasks, or architectures where attention poorly tracks grounding; richer region-level or architecture-agnostic temporal signals are the stated next step.
• Robustness to distractor and corrupted regions. Revisit can be misled by salient distractors or damaged image regions; patch dropout is already the hardest adversarial case in their evaluation.
• Closing the theory gap. Theorem 1 is an oracle variance analysis, and the surrogate bias from post-hoc revisit conditioning is only made explicit rather than eliminated — a principled correction is open.
• Extending the diagnostic. RAP is validated as method-agnostic here; using it as a general model-selection or early-stopping signal across RL recipes is a natural follow-up.

Target Audience

Researchers and engineers working on multimodal RL post-training, long chain-of-thought reasoning, or vision-language hallucination. It is most useful to practitioners who already run GRPO-style pipelines and want a low-overhead credit-assignment improvement, and to interpretability researchers interested in how cross-modal attention reorganizes during generation. Readers without RL or attention-mechanism background will find the empirical sections accessible but the formal sections (Proposition 1, Theorem 1, Appendix A/B) demanding.

Authors’ abstract

Long chain-of-thought reasoning improves vision-language models (VLMs), but it also exposes a temporal grounding failure: as generation unfolds, models may drift from image evidence while reinforcement learning still provides only a final-answer reward. We study this gap through visual revisit, a temporally localized reactivation of image evidence during reasoning. Across three VLM backbones, revisit peaks are predictive of correctness and causally linked to performance: masking them degrades accuracy more than masking non-peaks, while injecting revisit-like peaks into failed rollouts partially restores correct answers. Control analyses show that this signal is not explained by raw attention strength, uncertainty, logit margin, or hidden-state norm. We propose RECAP, a GRPO-compatible training method that turns detached revisit traces into a credit-assignment signal. RECAP uses revisit-conditioned advantage estimation, dynamic discounting, and gated value prediction to propagate final rewards through visually important reasoning steps, without extra rewards, additional rollouts, or inference-time changes. Across Qwen2.5-VL-7B, InternVL3-8B, and LLaVA-OV-7B, RECAP consistently improves over GRPO and VPPO, yielding $+6.1$--$+6.5$ HallusionBench gains over the SFT base while preserving reasoning and text-only performance. Results on larger LoRA-tuned models, Chinese zero-shot benchmarks, visual perturbations, and human evaluation further demonstrate robust improvements in image-grounded reasoning.