VIDEOP2R: Video Understanding from Perception to Reasoning

AI summary

Overview

Research area: Computer vision / multimodal machine learning — specifically reinforcement fine-tuning (RFT) of large video language models (LVLMs) for video reasoning and understanding.

Technical level: Advanced. The paper assumes familiarity with reinforcement learning for language models, group relative policy optimization (GRPO), chain-of-thought supervision, and video-language benchmarks.

Scope: The paper proposes VideoP2R, a two-stage reinforcement fine-tuning framework that separates video reasoning into explicit perception and reasoning processes, with a new chain-of-thought dataset (VideoP2R-CoT-162K) and a process-aware RL algorithm (PA-GRPO).

What This Paper Is About

Existing video RFT methods treat a model's entire output as one trajectory and assign a single final reward, so a perception mistake and a reasoning mistake are punished identically and credit cannot be assigned to the process that actually failed. VideoP2R addresses this by forcing the model to emit a separate perception segment (visual evidence) and a separate reasoning segment (inference over that evidence), then rewarding each segment with its own reward signal. The goal is to make video reasoning more accurate, more reliably trained, and more faithful to the visual input.

Key Contributions

1. A process-aware video RFT framework (VideoP2R) that models perception and reasoning as two distinct processes, using a standardized output template with <observation> (perception) and <think> / <answer> (reasoning) segments.

2. PA-GRPO, a process-aware extension of GRPO that supplies separate rewards for perception and reasoning tokens — an LLM-judged perception accuracy reward and a rule-based reasoning accuracy reward, plus per-process format and length rewards — and normalizes advantages within separate perception and reasoning groups.

3. VideoP2R-CoT-162K, a dataset built by an automatic three-step CoT generation pipeline (generation with Qwen2.5-VL-72B-Instruct, answer verification with task-specific metrics, and observation-sufficiency verification with Claude 3.7 Sonnet), yielding 162,062 process-aware annotated samples from 260K VQA pairs.

4. Comprehensive evaluation on seven video benchmarks plus ablations on two-stage training, process-aware modeling, and reward design, including analysis of perception sufficiency and PA-GRPO vs. GRPO training dynamics.

Main Findings

• State-of-the-art on six of seven benchmarks: VideoP2R (built on Qwen2.5-VL-7B-Instruct) reports VSI-Bench 36.8, VideoMMMU 55.0, MMVU 65.4, VCR-Bench 51.0, MVBench 68.1, TempCompass 74.5, VideoMME 60.0, with an average of 58.7. It ranks second on MMVU, where VideoRFT scores 68.5 versus VideoP2R's 65.4.

• Consistent improvement over the base model: The base Qwen2.5-VL-7B scores 30.1, 48.1, 60.0, 44.3, 59.0, 72.6, 56.6 (average 52.9), so VideoP2R gains 1.9% to 9.1% across benchmarks. The paper states VideoP2R surpasses the previous state of the art by 1.3% in average accuracy (VideoRFT's 57.4 average versus 58.7).

• Both training stages are needed: SFT-only reaches 55.6 average (+2.7 over the 52.9 baseline) and RL-only reaches 56.0 (+3.1), while combining both reaches 58.7 (+5.8).

• Process-aware modeling beats process-agnostic modeling: In SFT, the process-aware variant yields a 2.1% average accuracy gain over the process-agnostic variant; in RL, PA-GRPO leads on six benchmarks by 2.3% on average over GRPO (GRPO averages 56.4).

• Every reward component matters: Removing the reasoning reward drops the average to 55.3; removing the perception reward drops it to 56.4; removing the length reward gives 57.7; removing the separation of reward assignment gives 57.4. All fall short of the full 58.7.

• Length reward is not universally helpful: Removing the length reward improves VSI-Bench from 36.8 to 40.0. The paper attributes this to VSI-Bench questions needing long, fine-grained descriptions, and proposes a dynamic length reward as future work.

• Perception output carries sufficient information for reasoning: When Qwen2.5-VL-7B is given only the question text augmented with VideoP2R's perception segments, it reaches 55.5%, exceeding its accuracy under raw video input (52.9%). VideoP2R's perceptions consistently outperform Qwen's own perceptions in both text and video modality settings.

• PA-GRPO improves training efficiency: PA-GRPO exhibits consistently fewer samples suffering advantage collapse than GRPO during RL, and both perception and reasoning accuracy rewards increase during training.

• PA-GRPO mitigates Think-Answer Mismatch: Using Claude 3.7 Sonnet to extract answers from <think> segments and compare them to final answers, all SFT models maintain mismatch rates at or below 5%, while Video-R1 and VideoRFT's RL models show at least 16%. VideoP2R shows notably lower mismatch.

• Failure mode is knowledge, not perception: In the reported failure case, the model correctly identifies visual details but reaches a wrong conclusion because it lacks domain-specific knowledge (the paper's example is the molar volume of a gas being 22.4).

• Training cost: On 8× NVIDIA A100 GPUs, GRPO takes 16.5 hours for 1k steps and VideoP2R takes 18 hours.

Methodology in Plain English

The base model is Qwen2.5-VL-7B-Instruct. The authors first define an output template that separates perception from reasoning: everything about what the model sees goes in an <observation> block, and the inference plus final answer goes in <think> and <answer> blocks.

Stage 1 — SFT with generated data. They take 260K existing visual QA pairs and run a three-step pipeline. First, Qwen2.5-VL-72B-Instruct writes an initial perception-plus-reasoning trace for each sample. Second, the final answers are scored with task-specific metrics (exact match, word error rate, ROUGE variants, or regression error) and samples scoring below a threshold of 0.6 are discarded. Third, the <observation> segment alone — stripped of the video — is given to Claude 3.7 Sonnet along with the question and answer, to check whether the stated visual evidence is actually sufficient to support the answer. This produces 162,062 samples, and the base model is fine-tuned on them for one epoch.

Stage 2 — RL with process-aware rewards. They start from GRPO, which samples a group of responses per question, assigns each a reward, normalizes within the group to get an advantage, and applies a clipped policy-gradient update with a KL penalty against a frozen reference model. PA-GRPO changes two things. First, it decomposes the reward: perception tokens get an accuracy reward from an LLM judge (again Claude 3.7 Sonnet, judging <observation> sufficiency), while reasoning tokens get a task-specific accuracy reward. Both also get format rewards (checked with regular expressions) and length rewards, with perception responses targeted at 128–320 length units and reasoning responses at 320–512, with a length reward weight of 0.2. Second, the two reward streams are normalized in separate groups, because their scales are not comparable, and each advantage is applied only to its own tokens. The final answer text is the same in both; the difference is where the gradient signal lands.

Evaluation. They test on seven benchmarks: four video reasoning sets (VSI-Bench, VideoMMMU, MMVU, VCR-Bench) and three video understanding sets (MVBench, TempCompass, VideoMME), comparing against open-source 7B models (LLaVA-OneVision, LongVA, Video-UTR, VideoLLaMA2) and video-RFT baselines on the same base model (Video-R1, Time-R1, VideoRFT, VideoChat-R1, VersaVid-R1). Training used 16 video frames at 128×28×28 resolution; inference used 32 frames at 256×28×28 with top_p = 0.001 and temperature = 0.01.

Why This Matters

Impact on research. The paper challenges the default assumption in video RFT that a video is just another modality fed into a single-trajectory reward. By showing that separating perception from reasoning improves six of seven benchmarks, reduces reward stagnation, and lowers think-answer mismatch, it establishes process decomposition as a concrete design axis for multimodal RL. The released VideoP2R-CoT-162K dataset and its verification pipeline also provide reusable infrastructure for process-aware annotation.

Real-world applications (potential, not demonstrated in the paper):

• Video assistants and accessibility tools that must describe what is happening accurately before answering follow-up questions.
• Robotic or embodied agents where perception errors compound downstream into unsafe actions.
• Industrial and scientific video QA, such as procedures or equipment inspection, where separating "what was observed" from "what it implies" supports auditability.
• Educational or tutoring systems that need to show the evidence behind an answer rather than only the answer.

Industry relevance. Video understanding is a core capability for large multimodal product pipelines, and the paper demonstrates a training recipe that is implemented on 8× A100 GPUs and costs only 18 hours for 1k RL steps — a practical scale. The explicit separation of evidence and inference also maps naturally onto requirement-driven deployment contexts where output traceability matters.

Future Directions

• Dynamic length rewards. The authors explicitly propose this after finding that a fixed length reward hurts VSI-Bench, where long fine-grained descriptions are needed for sufficient perception.
• Injecting factual and domain-specific knowledge. The reported failure case shows the model perceiving correctly but answering incorrectly due to missing knowledge; closing that gap is an open problem for the authors.
• Addressing the MMVU gap. VideoP2R is second rather than first on MMVU, which the paper attributes to missing domain-specific knowledge (for example, chemistry) in the training data.
• Extending process-aware modeling to other domains. The related-work section notes that separation of perception and reasoning remains largely unaddressed in the image domain, suggesting process-aware rewards as a direction beyond video.

Target Audience

Researchers and practitioners working on multimodal large language models, reinforcement learning from verifiable rewards, and video-language understanding. It is most useful to readers who already understand GRPO-style policy optimization and chain-of-thought supervision, and who want a concrete, evaluated design for decomposing rewards across distinct cognitive stages of a model's output. Engineers building video QA systems will find the training cost, data-construction pipeline, and reward design specifics directly actionable.

Authors’ abstract

Reinforcement fine-tuning (RFT), a two-stage framework consisting of supervised fine-tuning (SFT) and reinforcement learning (RL) has shown promising results on improving reasoning ability of large language models (LLMs). Yet extending RFT to large video language models (LVLMs) remains challenging. We propose VideoP2R, a novel process-aware video RFT framework that enhances video reasoning by modeling perception and reasoning as distinct processes. In the SFT stage, we develop a three-step pipeline to generate VideoP2R-CoT-162K, a high-quality, process-aware chain-of-thought (CoT) dataset for perception and reasoning. In the RL stage, we introduce a novel process-aware group relative policy optimization (PA-GRPO) algorithm that supplies separate rewards for perception and reasoning. Extensive experiments show that VideoP2R achieves state-of-the-art (SotA) performance on six out of seven video reasoning and understanding benchmarks. Ablation studies further confirm the effectiveness of our process-aware modeling and PA-GRPO and demonstrate that model's perception output is information-sufficient for downstream reasoning. Our project page is available at https://videop2r.github.io/videop2r/.