Look as You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning

AI summary

Overview

Research area: Multimodal retrieval-augmented generation over visual documents, specifically visual evidence attribution in vision-language models (VLMs), combined with reinforcement learning for reasoning.

Technical level: Advanced. The paper assumes familiarity with VLMs, chain-of-thought reasoning, LoRA fine-tuning, GRPO-style reinforcement learning, bounding-box grounding, and retrieval metrics such as IoU and exact match.

Scope: The paper proposes Chain-of-Evidence (CoE), a reasoning paradigm that interleaves chain-of-thought steps with bounding-box and page-index attribution to source documents, and Look As You Think (LAT), a two-stage (supervised fine-tuning then reinforcement learning) framework that trains VLMs to produce such verifiable reasoning paths for visual document RAG (VD-RAG).

What This Paper Is About

Vision-language models answering questions over document images can hallucinate, and existing attribution methods link only the final answer to a source region without showing how the model got there, which makes verification hard. This paper introduces Chain-of-Evidence (CoE), which grounds each intermediate reasoning step in a specific bounding box and page index, and LAT, a reinforcement learning framework that rewards trajectories only when each step's attributed evidence is semantically consistent with the step's text and the whole trajectory ends in a correct answer.

Key Contributions

1. The Chain-of-Evidence (CoE) paradigm. In the VD-RAG setting, multimodal reasoning is formalized as a sequence of grounded steps, where each reference element (figure, table, or factual information) is linked to its source through a bounding box and a page index, producing a coarse-to-fine "chapter–paragraph summary–specific element" observation path.

2. The LAT framework. A two-stage approach: Stage I fine-tunes a VLM on a small set of human-verified CoE annotations, and Stage II applies reinforcement learning under Group Relative Policy Optimization (GRPO) with a tailored rule-based reward design.

3. A stepwise, answer-conditioned attribution reward. The reward evaluates semantic alignment between each predicted evidence crop and its reasoning step using the ColQwen2 multimodal retriever, penalizes redundant overlapping boxes via pairwise IoU, and is gated so that step rewards are only issued when the CoE trajectory yields a correct answer, enabling process-level self-verification without stepwise human annotations.

4. Demonstrated balance of traceability and performance. Trained on vanilla Qwen2.5-VL-7B-Instruct, LAT improves over the vanilla model and outperforms a supervised fine-tuning baseline trained to directly produce answers with attribution, with stronger cross-domain generalization.

Main Findings

• Answer and grounding gains over the vanilla model: Across Paper- and Wiki-VISA in single- and multi-image settings, the abstract reports average improvements of 8.23% in soft exact match (EM) and 47.0% in IoU@0.5 over vanilla Qwen2.5-VL-7B-Instruct. Section 5 states the comparison as +7.95% EM and +44.6% IoU@0.5; both figures are reported in the paper as written.

• In-domain results (LAT-Ind.): Wiki-VISA single-image 73.6 EM / 53.7 IoU@0.5; Paper-VISA single-image 45.4 / 49.9; Wiki-VISA multi-image 64.5 / 38.0; Paper-VISA multi-image 51.2 / 46.9. Corresponding deltas versus the vanilla model are +4.20 EM / +57.0 IoU@0.5 (Wiki single), +2.40 / +47.0 (Paper single), +12.3 / +40.4 (Wiki multi), and +14.0 / +43.7 (Paper multi).

• Cross-domain results (LAT-Full, trained on sampled subsets from all datasets): Wiki-VISA single 73.1 / 57.8, Paper-VISA single 46.2 / 48.4, Wiki-VISA multi 64.8 / 41.4, Paper-VISA multi 50.6 / 49.3, improving over LAT-Ind. on several settings (for example Wiki-VISA multi IoU@0.5 from 38.0 to 41.4, and Stepwise Attribution from 71.8 to 75.2).

• Stepwise attribution quality: Under CoE-style evaluation, LAT-Ind. reaches Stepwise Attribution (SA) scores of 64.6 (Wiki single), 35.5 (Paper single), 71.8 (Wiki multi), and 46.3 (Paper multi), versus vanilla scores of 13.9, 12.4, 12.5, and 29.5. One-shot in-context learning yields only a moderate SA improvement of +4.0%, while LAT delivers an SA gain of 37.5%. The paper reports an average SA of 57.1% for LAT.

• Low-resource training: LAT is trained on only 5% of raw QA pairs during the RL stage, whereas VISA-7B relies on large-scale (100k) supervised data. In this low-resource setting LAT achieves comparable answer accuracy and attribution precision to VISA-7B and, on high-resolution Wiki-VISA images, outperforms it.

• Robustness on unanswerable cases: For unanswerable questions, LAT outputs "No answer" and achieves an average precision of 65%.

• Cross-domain transfer: Transferring from Paper-VISA to Wiki-VISA, LAT-Ind. outperforms an SFT baseline by 1.7% EM (67.7 vs. 66.0) and 6.2% IoU@0.5 (35.6 vs. 29.4) in the single-image setting, and by 8.6% EM (57.3 vs. 48.7) and 11.1% IoU@0.5 (21.4 vs. 10.3) in the multi-image setting.

• Cold-start stage value: The Stage I model M_dist improves over the vanilla model by an average of 6.48% EM and 20.9% IoU@0.5.

• Ablation on the stepwise reward: Removing the stepwise attribution reward R_step causes a 15.5% reduction in SA and also lowers IoU@0.5, since answer grounding depends on the quality of intermediate attribution. Without the pairwise overlap constraint, the model tends to reuse large redundant regions across steps.

• Ablation on jointly optimizing process and result objectives: When R_acc and R_ground are removed and used only as a filter, EM remains moderate, suggesting that consistency between reasoning and answer provides useful training signal. When the R_acc ≥ ε condition is also removed from R_step, the objective reduces to format alignment and both accuracy and grounding degrade substantially.

• Threshold sensitivity: The default attribution threshold is τ = 0.3; a high threshold (τ = 0.5) fails to sustain gains because positive instances become hard to sample, and a no-step variant (equivalent to τ = 0 or 1, where all steps are uniformly rewarded) underperforms due to the lack of fine-grained attribution guidance.

Methodology in Plain English

The researchers start from the observation that people answering questions from documents do not jump straight to the answer; they narrow down from chapter to paragraph to a specific element. They encode this as Chain-of-Evidence: the model writes reasoning steps and, for each step that references something in the document, emits a bounding box and the page index of the supporting region.

Training happens in two stages. In Stage I (cold start), they sample 1,000 instances from each training dataset, prompt the proprietary model Gemini 2.5 Pro with two CoE exemplars as in-context demonstrations, and keep responses whose answers overlap enough with ground truth, as measured by a recall metric with threshold γ. They manually correct bounding-box drift and retain verified samples with correct answers (~30%), then fine-tune the VLM with LoRA on this set, producing a model called M_dist.

In Stage II, they initialize from M_dist and run GRPO reinforcement learning on 5% of the raw QA pairs. The reward combines four pieces: an accuracy reward averaging exact match and recall; a stepwise attribution reward that crops each predicted bounding box, encodes the crop and the reasoning text with the ColQwen2 multimodal retriever, requires the minimum cosine similarity across all step pairs to exceed a threshold τ, requires the maximum pairwise IoU among boxes to stay below a threshold, and is zeroed out unless the accuracy reward meets a threshold ε; a grounding reward given when the answer's bounding box has IoU above 0.5 with the ground-truth evidence on the correct page; and a format reward of +1 for correct <think>/<answer> tagging and −1 otherwise. Hyperparameters reported include LoRA rank r = 64 and scaling factor α = 64, SFT learning rate 1e-4 and RL learning rate 5e-5, one RL epoch, 8 rollouts for Paper-VISA and 6 for Wiki-VISA, and maximum prompt and completion lengths of 16,384 and 600 tokens. Evaluation is done with greedy decoding on Wiki-VISA, Paper-VISA, and (for training data) FineWeb-VISA, measuring soft EM, IoU@0.5, and Stepwise Attribution.

Why This Matters

For research, the paper reframes visual evidence attribution from a single final answer-to-region link into a process-level, step-by-step grounding problem, and shows that rule-based RL rewards can supervise intermediate attribution without stepwise human annotations. It also demonstrates that process rewards and outcome rewards are complementary rather than interchangeable, and that cross-domain generalization improves when both are optimized jointly.

Real-world applications:

• Document question answering over scanned or screenshot PDFs, where a user needs to see exactly which figure, table, or sentence supports an answer in financial filings, scientific papers, or technical manuals.
• Regulatory and compliance review, where an auditor must trace a conclusion back to a specific region of a source page.
• Medical or scientific literature assistants, where citations must point to the correct panel of a figure or a specific table cell rather than the whole article.
• Customer support and enterprise search over heterogeneous documents, where multi-page layouts and retrieved candidate pages must be disambiguated before an answer is trusted.

Industry relevance: the method uses a 7B open-weight backbone (Qwen2.5-VL-7B-Instruct) with LoRA adapters and trains on only 5% of raw QA pairs, which points to a tractable path for deploying verifiable document assistants without the full-scale annotation budgets used by prior attribution-supervised models such as VISA. The released code at github.com/PolarisLiu1/LAT supports reproduction.

Future Directions

• Determining how far the approach scales to larger backbones and to document collections beyond Wiki-VISA, Paper-VISA, and FineWeb-VISA, given that the paper's reported experiments use Qwen2.5-VL-7B-Instruct.
• More principled tuning or automatic selection of the attribution threshold τ and the overlap constraint, since results vary between the default τ = 0.3 and the high-threshold τ = 0.5 setting.
• Extending CoE beyond bounding-box and page-index evidence to other document modalities or formats not covered by the VISA benchmark.
• Evaluating the approach on real user verification workflows, since the paper's evaluation relies on EM, IoU@0.5, and the Stepwise Attribution reward rather than human studies of traceability in use.

Target Audience

Researchers and engineers working on multimodal RAG, document understanding, and vision-language model alignment; practitioners building verifiable or citation-backed document assistants; and readers interested in reinforcement learning reward design for grounding intermediate reasoning steps. Readers need a working knowledge of VLMs, chain-of-thought prompting, and policy-gradient style RL to follow the reward formulation and experimental tables.

Authors’ abstract

Aiming to identify precise evidence sources from visual documents, visual evidence attribution for visual document retrieval-augmented generation (VD-RAG) ensures reliable and verifiable predictions from vision-language models (VLMs) in multimodal question answering. Most existing methods adopt end-to-end training to facilitate intuitive answer verification. However, they lack fine-grained supervision and progressive traceability throughout the reasoning process. In this paper, we introduce the Chain-of-Evidence (CoE) paradigm for VD-RAG. CoE unifies Chain-of-Thought (CoT) reasoning and visual evidence attribution by grounding reference elements in reasoning steps to specific regions with bounding boxes and page indexes. To enable VLMs to generate such evidence-grounded reasoning, we propose Look As You Think (LAT), a reinforcement learning framework that trains models to produce verifiable reasoning paths with consistent attribution. During training, LAT evaluates the attribution consistency of each evidence region and provides rewards only when the CoE trajectory yields correct answers, encouraging process-level self-verification. Experiments on vanilla Qwen2.5-VL-7B-Instruct with Paper- and Wiki-VISA benchmarks show that LAT consistently improves the vanilla model in both single- and multi-image settings, yielding average gains of 8.23% in soft exact match (EM) and 47.0% in IoU@0.5. Meanwhile, LAT not only outperforms the supervised fine-tuning baseline, which is trained to directly produce answers with attribution, but also exhibits stronger generalization across domains.