TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learning

AI summary

Overview

Research area: Computer vision and multimodal large language models, specifically long-form video understanding and adaptive temporal search.

Technical level: Advanced. The paper assumes familiarity with reinforcement learning for language models (GRPO), vision-language models, and video tokenization, though its core idea — letting a model decide where to look in a video — is easy to grasp.

Scope: The paper introduces TimeSearch-R, a Qwen2.5-VL-7B-Instruct–based model trained with a modified reinforcement learning algorithm (GRPO-CSV) that performs temporal search as an interleaved text-video reasoning process, and reports results on temporal search benchmarks (Haystack-LVBench, Haystack-Ego4D) and long-form video understanding benchmarks (VideoMME, MLVU, LongVideoBench).

What This Paper Is About

Long videos contain tens of thousands of frames, and a model must find the small handful of frames that actually answer a question. Existing systems either use fixed frame sampling decided before reasoning begins, or hand-built agent pipelines that call tools such as captioners and object detectors in a manually designed sequence. The authors argue that this is backwards — reasoning should be dynamic, so the frames the model can see should change as the reasoning progresses. TimeSearch-R reformulates temporal search as an interleaved text-video "thinking" process and uses reinforcement learning to discover the search strategy directly from data instead of hand-designing it.

Key Contributions

1. The TimeSearch-R framework, which reformulates temporal search as interleaved text-video thinking — the model alternates between textual reasoning steps and search instructions that retrieve video clips, calling this paradigm "Thinking with Videos."
2. GRPO-CSV, a reinforcement learning algorithm that extends Group Relative Policy Optimization (GRPO) with a Completeness Self-Verification stage: the same policy model must re-answer the question using only the frames it searched for, providing annotation-free supervision on intermediate search decisions.
3. A high-quality video reasoning dataset built through a two-stage filtering pipeline that removes samples solvable by linguistic shortcuts and samples that stay unsolvable even with extensive search, drawing on Haystack-Ego4D, VideoMarathon (Panda-70M), and CinePile.
4. Empirical validation showing improvements on temporal search and long-form video understanding, including a reported new state-of-the-art on LongVideoBench (60.1%), 4.1% above the base model Qwen2.5-VL and 2.0% above Video-R1.

Main Findings

• Temporal search quality: On Haystack-LVBench with a budget of 8 keyframes, TimeSearch-R reaches a temporal similarity F1 of 8.1 (precision 5.4, recall 22.3), stated to be more than three times the previous best result of 2.5 from T*. Visual similarity F1 reaches 69.2 (precision 63.2, recall 76.4), which the paper says surpasses the previous SOTA method VideoAgent by 5.5.
• Needle-in-a-haystack QA: TimeSearch-R achieves 52.1% accuracy on Haystack-LVBench and 53.5% on Haystack-Ego4D, outperforming GPT-4o. The abstract reports gains of 5.6% temporal F1 on Haystack-LVBench and 8.5% accuracy on Haystack-Ego4D over the previous state of the art.
• Long-form video understanding: VideoMME overall accuracy is 66.6% (short 76.8, medium 67.1, long 56.0), a 1.5% gain over the base Qwen2.5-VL-7B, with gains growing as videos lengthen (+0.5% short, +1.4% long). MLVU reaches 71.5% (+1.3%) and LongVideoBench reaches 60.1% (+4.1%).
• Advantage over agents and text-only reasoning: TimeSearch-R outperforms VideoAgent and T* on VideoMME by 10.6% and 2.5% respectively, and consistently surpasses Video-R1 across all benchmarks.
• Naive prompting hurts: Applying temporal search to Qwen2.5-VL through CoT prompting without training degrades performance (51.8% overall on VideoMME versus 65.1% for untrained Qwen2.5-VL), underscoring the need for RL post-training.
• SFT gives search ability; RL gives reasoning quality: Supervised fine-tuning on the cold start lifts temporal F1 from 0.0 to 7.8 and search completeness from 44.2% to 60.5%. RL mainly improves answer quality, raising consistency by 2.6% and QA accuracy from 59.2% to 66.6%.
• Plain GRPO collapses: Without the CSV component, completeness drops from 60.5% to 57.2% and temporal F1 from 7.8 to 7.4; training collapses around step 300, after which the model stops issuing search calls entirely and completeness falls to zero.
• Ablation of reward components: CSV without the accuracy reward achieves the highest completeness (61.2) and consistency (75.3) but slightly lowers QA accuracy by 0.3%; combining CSV with the accuracy reward gives the best overall QA accuracy (66.6%).
• Data filtering is essential: Training without filtering drops overall accuracy to 62.8% (worse than the untrained 65.1% baseline) because linguistic shortcuts produce zero advantage within GRPO groups. Egocentric-only data recovers baseline performance (65.3%), while adding exocentric data yields the best general QA accuracy (66.6%).
• Generalization to untrained skills: Although training uses only general long-video QA, RL raises temporal reasoning by 7.4% and action reasoning by 5.7%.
• Emergent search behaviors: Analysis identifies hypothesis-driven search, confirmation or elimination, and sequential segment-by-segment search patterns.

Methodology in Plain English

The model is given a question and a small uniformly sampled preview of the video. It then produces a chain of thought: some thinking text, then optionally a search instruction that specifies a time window and a text query. A search function retrieves a fixed number of frames (up to 8) from that window by embedding frames with a small vision model such as SigLIP, scoring relevance to the query, and selecting a diverse, relevant subset using a determinantal point process (DPP) with fast greedy MAP inference. Each retrieved frame carries an explicit absolute timestamp in seconds. This loop continues until the model emits an answer or hits the reasoning budget (up to 8 search steps).

Training happens in two stages. First, supervised fine-tuning on reasoning traces generated by GPT-4o teaches the output format and search behavior, with search results masked so the model must learn meaningful time windows and queries. Second, reinforcement learning with GRPO-CSV:

• The total reward is the sum of three parts: a completeness reward, a format reward, and an accuracy reward.
• The completeness reward comes from the self-verification phase. The frames collected during reasoning are pooled into a dynamic frame set, and the same policy model must re-answer the question using only those frames — no further searching allowed. The reward is the accuracy of that second answer, gated by an indicator that fires only when the original answer was already correct.
• The format reward checks that each step follows either the <think>...</think><tool_call>...</tool_call> pattern or the <think>...</think><answer>...</answer> pattern, scored as a binary 1 or 0 over the whole trajectory.
• The accuracy reward is exact match for multiple-choice questions and an LLM-as-a-Judge comparison for open-ended questions.

Training details: AdamW optimizer, learning rate 1e-6, KL penalty coefficient β = 0.005, batch size of 4 with 8 rollouts per prompt, 32 A100 GPUs. The dataset has a mean video length of 1,659 seconds with a tail beyond 10,000 seconds; 49.5% of samples come from Ego4D (Haystack-Ego4D), 35.6% from Panda-70M (VideoMarathon), 9.5% from CinePile and 5.4% from other sources. Questions are 60.3% open-ended and 39.7% multiple choice. Human annotation was used to clean VideoMarathon's automatically generated QA pairs.

Why This Matters

Impact on research: The paper moves temporal video search from hand-crafted agent workflows to end-to-end policy learning, and shows that outcome-only RL rewards are not enough for video — intermediate search decisions need their own supervision signal. The self-verification trick (re-answer with only the retrieved frames) is an annotation-free way to get that signal, which matters because frame-level temporal annotations are expensive. It also extends the "thinking with images" line of work into the video domain.

Real-world applications:

• Searching long security, traffic, or industrial camera footage for a described event, where reviewing tens of thousands of frames manually is impractical.
• Wearable and egocentric assistants, since the paper trains and evaluates on Ego4D data with frequent viewpoint changes in daily activities.
• Sports or coaching analysis and instructional video review, where a question refers to a specific moment inside a long recording.
• Media and archive retrieval, where a user asks a natural-language question about a long film or broadcast and needs the relevant seconds returned.

Industry relevance: The work comes from ByteDance, a company with large-scale short- and long-video platforms, and targets exactly the cost problem those platforms face: rather than feeding a model hundreds of frames, adaptive search promises to retrieve a small set of informative frames (the model averages 8.8 keyframes on Haystack-LVBench). A more capable video QA model also underpins video search, content moderation, recommendation metadata, and video editing assistants.

Future Directions

• Scaling and architecture: Only a 7B-parameter base model is studied (Qwen2.5-VL-7B-Instruct). Whether the learned search strategies and GRPO-CSV gains transfer to larger and other model families is not reported.
• Extending beyond visual search: The framework searches over frames only. Applying the same interleaved reasoning to audio, speech, or text-in-video (subtitles, on-screen text) is an open question.
• Making verification more informative: The completeness reward is gated on the original answer being correct, and CSV without the accuracy reward actually lowers QA accuracy by 0.3%. Better or denser intermediate rewards, and reducing the reliance on the accuracy component, remain open.
• Cost and efficiency: The paper lists an efficiency analysis appendix section, but the details are not included in the provided content. How much search-time computation is saved relative to uniform sampling at scale, and how the approach behaves under tighter compute budgets, are natural follow-ups.
• Generalization to untrained domains: The observed transfer to temporal (+7.4%) and action (+5.7%) reasoning despite training only on general QA suggests more systematic study of what cognitive patterns RL actually teaches.

Target Audience

Researchers and engineers working on video-language models, multimodal reasoning, and reinforcement learning for LLM post-training. It is most useful for those building long-video QA systems, agentic video retrieval, or RL pipelines with tool calls, and for practitioners at video platforms who care about the compute cost of processing long videos. Readers without a background in RL or vision-language modeling will find the high-level ideas accessible but the algorithm details demanding.

Authors’ abstract

Temporal search aims to identify a minimal set of relevant frames from tens of thousands based on a given query, serving as a foundation for accurate long-form video understanding. Existing works attempt to progressively narrow the search space. However, these approaches typically rely on a hand-crafted search process, lacking end-to-end optimization for learning optimal search strategies. In this paper, we propose TimeSearch-R, which reformulates temporal search as interleaved text-video thinking, seamlessly integrating searching video clips into the reasoning process through reinforcement learning (RL). However, applying RL training methods, such as Group Relative Policy Optimization (GRPO), to video reasoning can result in unsupervised intermediate search decisions. This leads to insufficient exploration of the video content and inconsistent logical reasoning. To address these issues, we introduce GRPO with Completeness Self-Verification (GRPO-CSV), which gathers searched video frames from the interleaved reasoning process and utilizes the same policy model to verify the adequacy of searched frames, thereby improving the completeness of video reasoning. Additionally, we construct datasets specifically designed for the SFT cold-start and RL training of GRPO-CSV, filtering out samples with weak temporal dependencies to enhance task difficulty and improve temporal search capabilities. Extensive experiments demonstrate that TimeSearch-R achieves significant improvements on temporal search benchmarks such as Haystack-LVBench and Haystack-Ego4D, as well as long-form video understanding benchmarks like VideoMME and MLVU. Notably, TimeSearch-R establishes a new state-of-the-art on LongVideoBench with 4.1% improvement over the base model Qwen2.5-VL and 2.0% over the advanced video reasoning model Video-R1. Our code is available at https://github.com/Time-Search/TimeSearch-R.