LongVPO: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization

AI summary

Overview

Research area: Computer vision and multimodal large language models — specifically long-form video understanding and preference-based post-training (Direct Preference Optimization, DPO).

Technical level: Advanced. The paper assumes familiarity with vision-language models, DPO objectives, positional encoding extension methods (YARN, NTK), and long-context training infrastructure.

Scope: The paper proposes LongVPO, a two-stage DPO framework that turns short-context vision-language models into long-video understanding systems using about 16K synthetic preference examples and no long-video annotations.

What This Paper Is About

Vision-language models handle images and short clips well, but their performance degrades on videos that last tens of minutes to over an hour. Fixing this usually requires expensive, high-quality long-video annotations, and the paper shows another obstacle: short-context models exhibit a "lost-in-the-middle" positional bias, where content near the center of a long sequence is answered worse than content at the edges. LongVPO's goal is to extend a short-context model into the ultra-long-video regime using only synthetic preference data built from short clips and unlabeled long videos.

Key Contributions

1. A two-stage DPO framework (LongVPO) that extends short-context VLMs to long-video contexts without any long-video annotations, and without proprietary annotation models.
2. A synthetic training set of roughly 16K instances (10k for Stage 1 from short-clip data, 6k for Stage 2 from unlabeled long videos), described as significantly fewer than existing instruction-tuning datasets.
3. An anchor-only approximation of the reference model's likelihood, which evaluates the frozen reference model on the anchor clip rather than the full composite sequence, avoiding context-length mismatch and reducing computational overhead (reported as requiring approximately 70% of the training time of the alternative).
4. A recursive captioning and self-training pipeline for Stage 2 that produces multi-segment reasoning queries and dispreferred responses using scene-level metadata, rather than human labels.

Main Findings

• Long-video benchmark gains over baselines. With InternVL2.5-8B (64-frame baseline: LVBench 43.2, LongVideoBench 60.0, MLVU 68.9, Video-MME 64.2/66.9), LongVPO Stage 2 reaches LVBench 50.1 (+4.9), LongVideoBench 66.6 (+3.9), MLVU 74.1 (+6.5), and Video-MME 64.6/70.3 (+3.5/+5.0).
• Both stages contribute. Stage 1 alone on the same base gives 49.4 (+4.2) on LVBench, 65.4 (+2.7) on LongVideoBench, and 73.5 (+5.9) on MLVU; Stage 2 adds further improvement in most settings.
• Short-video performance is preserved. On MVBench, LongVPO reaches 73.1 with InternVL2.5-8B versus 72.0 for the baseline, which the paper describes as a +1.1 improvement that surpasses prior results.
• Generalization to a dedicated long-video model. Applied to InternVideo2.5 (baseline LVBench 47.4, LongVideoBench 63.2, MLVU 72.8), LongVPO reaches 51.0 (+3.6), 67.2 (+4.0), and 74.7 (+1.9) after Stage2-iter2, indicating the base model had not saturated.
• Scaling with frame count. Performance improves as the maximum number of training frames increases (LongVPO-256f to LongVPO-512f), while the baseline model shows signs of saturation.
• A positional bias exists in short-context models. Embedding a short video in a padded, longer sequence (visualized as a 4×4 grid) produces a performance dip for content near the center, measured by L1 distance from the query token — a "lost-in-the-middle" effect. The appendix reports the same phenomenon with a 10×10 padding grid instead of 12×12.
• Long-context robustness in needle-in-a-haystack testing. The InternVL2.5 baseline begins significant degradation at around 800 frames and fails to follow instruction output formats at approximately 1k frames (evaluated up to about 3k frames), while LongVPO maintains better long-context modeling.
• Filtering matters. In Stage 1, using scene filtering gives MLVU 72.9, LongVideoBench 66.1, LVBench 45.3; removing it with a similar clip added drops to 69.8, 64.2, 43.4, and Top-K selection instead gives 69.9 and 58.4 on MLVU and LongVideoBench.
• Self-generated preferred responses work best. For the chosen response in Stage 2, self-generated responses give 72.9/66.1/45.3 (MLVU/LongVideoBench/LVBench); LLM-generated responses from Qwen2.5-32B give 73.1/65.6/44.4, and scene-interleaved captions give 73.0/66.1/44.7.
• The framework does not require a large LLM. Replacing Qwen2.5-32B with the 7B InternLM2.5 backbone yields 72.5/65.8/44.9 — only a slight drop.
• SFT on single-video captions hurts; DPO helps most. Fine-tuning on single-video caption data leads to a noticeable performance drop attributed to overfitting on SFT-seen data, SFT on the synthesized Stage 1 data reverses the trend, and DPO on the preference pairs gives the largest improvements.
• Qualitative superiority in a counting task. In a pumpkin-carving action-counting test, Qwen2.5-VL, Qwen2-VL, and LLaVA-Video failed to produce correct counts, while LongVPO identified all 5 instances.
• Video-LLaMA3 also benefits (appendix Table 3). From a base of LVBench 45.3, LongVideoBench 59.8, MLVU 73.0, Video-MME 66.2/70.3, LongVPO improves these to 49.8 (+4.5), 63.4 (+3.6), 74.6 (+1.6), and 67.2/71.4 (+1.0/+1.3).

Methodology in Plain English

The framework trains in two stages, both built on DPO, which teaches a model by comparing a preferred answer against a dispreferred one.

Stage 1 — anchored cues. Take short clips with captions from LLaVA-Video-178K. Pick one clip as the "anchor" and have a model generate a question that can only be answered from that clip; its answer becomes the preferred response. Then concatenate the anchor with several other clips into a fake long video and generate a dispreferred answer by prompting with non-anchor clips, simulating a mislocated answer. Two filters keep supervision clean: clips whose DINO embedding cosine similarity with the anchor exceeds 0.6 are discarded, and an optional LLM (Qwen-2.5-32B) checks that questions truly depend on multiple visual elements of the anchor. Because the frozen reference model cannot handle long inputs, the authors approximate its score using only the anchor clip. Clips are sampled at up to 64 frames at 1 fps, and the anchor's position in the composite sequence is randomized to fight positional bias.

Stage 2 — self-reasoning on real long videos. Using scene-segmented but unlabeled videos from Vript, a recursive captioning pipeline describes each scene conditioned on earlier scene captions. An external LLM (Qwen2.5-32B) then writes queries requiring cross-scene reasoning plus a reasoning trace that cites specific scene IDs, effectively providing binary scene-question relevance labels. The model answers these queries watching the full video (preferred response), and dispreferred responses are induced by showing it only part of the relevant scenes or only irrelevant scenes. Stage 2 uses the standard DPO objective, initialized from the Stage 1 checkpoint, with the Stage 1 checkpoint serving as the frozen reference. In both stages an SFT loss on the preferred response, weighted by a coefficient alpha, is added to the DPO loss.

Setup. The base model is InternVL-2.5-8B (InternViT-300M vision encoder plus InternLM-2.5-7B-32K language backbone), originally trained on a maximum of about 32 video frames (roughly 8192 visual tokens). Training uses DeepSpeed Ulysses sequence parallelism for a 32K extended video context on a server with 4 × 8 NVIDIA H100 GPUs, each with 80GB of memory.

Why This Matters

Impact on research. The paper argues that long-video capability can be bootstrapped from short-clip data and unlabeled long videos rather than from costly long-video annotation pipelines. It also documents an explicit positional bias in short-context VLMs and shows that DPO with a reference model evaluated on just the anchor clip is both cheaper and more effective than feeding the reference model full long contexts.

Real-world applications:

• Long-form video assistants that summarize or answer questions about hour-long recordings such as lectures, meetings, or commentaries.
• Video surveillance and monitoring review, where relevant events are brief but embedded in very long recordings.
• Sports and broadcast analysis, including temporal tasks such as counting repeated actions across a full event.
• Media and content archives, where search and QA must work across

Authors’ abstract

We present LongVPO, a novel two-stage Direct Preference Optimization framework that enables short-context vision-language models to robustly understand ultra-long videos without any long-video annotations. In Stage 1, we synthesize preference triples by anchoring questions to individual short clips, interleaving them with distractors, and applying visual-similarity and question-specificity filtering to mitigate positional bias and ensure unambiguous supervision. We also approximate the reference model's scoring over long contexts by evaluating only the anchor clip, reducing computational overhead. In Stage 2, we employ a recursive captioning pipeline on long videos to generate scene-level metadata, then use a large language model to craft multi-segment reasoning queries and dispreferred responses, aligning the model's preferences through multi-segment reasoning tasks. With only 16K synthetic examples and no costly human labels, LongVPO outperforms the state-of-the-art open-source models on multiple long-video benchmarks, while maintaining strong short-video performance (e.g., on MVBench), offering a scalable paradigm for efficient long-form video understanding.