VideoGen-Agent: Reinforcing Video Generation Agents

AI summary

Overview

• Research area: Computer Vision / video generation, specifically agentic reinforcement learning for tool-augmented video generation.
• Technical level: Advanced. The paper assumes familiarity with diffusion-based video generation, supervised fine-tuning, GRPO-style policy optimization, and multimodal agent architectures.
• Scope: The paper trains a single multimodal agent policy to coordinate retrieval, simulation, generation, and verification tools across six video-generation capability categories, and introduces the VABench held-out benchmark to evaluate the result.

What This Paper Is About

Current video generators produce visually convincing footage but often fail to follow what a prompt actually specifies: they may miss entities or events that emerged after pretraining, lose a person's visual identity mid-clip, violate physical laws, or depict steps out of order. The authors train one multimodal agent, VideoGen-Agent, to close that gap by deciding which external tools to call at each step, using the observations those tools return to guide the next decision. The goal is a single shared policy that handles six distinct video-generation challenges rather than one narrowly specialized pipeline per task.

Key Contributions

1. VideoGen-Agent, a multimodal agent trained through multitask agentic reinforcement learning. SFT on teacher-distilled trajectories followed by multitask RL trains a shared policy to coordinate augmentation, generation, and verification tools.
2. A tool-use trajectory dataset spanning six task categories, built from prompts generated with Claude Opus 4.7 and supplemented from public datasets, with trajectories produced by Gemini 3.1 Pro or Claude Opus 4.7 teachers. It totals 24K trajectories, split into 16K for SFT and 8K for RL.
3. VABench, a held-out benchmark of 600 prompts with 100 prompts per category, covering Procedural Knowledge, Single-Entity Identity, Multi-Entity Identity, Physics Simulation, Compositional Scene, and Multi-Shot, scored by category-specific VLM rubrics validated against human preferences.
4. Evidence that the trained policy transfers to upgraded tools: replacing generation tools without additional agent training raises the VABench score from 75.6 to 86.1, with the highest score in every category.

Main Findings

• Overall improvement over the base generator: With Toolset 1, VideoGen-Agent scores 75.6 on VABench, a 19.1-point gain over its base T2V generator Seedance 1.0 Pro Fast at 56.5. It also exceeds Seedance 2.0, the strongest standalone baseline at 73.2 overall, by 2.4 points.
• Tool upgrades compound with learned tool use: Toolset 2 reaches 86.1 overall and the highest score in all six categories, with the agent policy fixed and augmentation and verification tools unchanged. The largest single jump is Multi-Entity Identity, from 65.3 to 86.7.
• Category-level gains vary by capability: Relative to Seedance 2.0, Toolset 2 improves Procedural Knowledge by 14.1 points, Multi-Entity Identity by 17.7, and Multi-Shot by 25.7; gains on Single-Entity Identity and Physics Simulation are 8.2 and 9.0 points; Compositional Scene improves by 2.9 points, where the strongest baseline already scores 87.8.
• Human raters prefer the agent: In a side-by-side evaluation on 100 randomly sampled video pairs from VABench comparing Toolset 2 against Seedance 2.0, four human raters gave the agent an overall preference rate of 84.3 percent, with the largest margins on Procedural Knowledge and Multi-Shot.
• Both training stages matter: Prompt rewriting raises the score only from 56.5 to 57.3, and zero-shot tool use reaches 59.5. SFT raises it to 69.2, and RL adds another 6.4 points to 75.6 with the toolset held fixed. Every category improves at both stages.
• Both reward components contribute: Removing the VLM reward lowers the score to 73.3, and removing the tool reward lowers it to 71.2, with both ablations reducing performance across all six categories.
• Multitask RL is close to per-task training: Single-task RL scores 76.3 overall, exceeding shared multitask RL by 0.7 points and doing better on Procedural Knowledge, Multi-Entity Identity, and Compositional Scene, while multitask RL performs better on the remaining categories.
• The agent extends to new tool pipelines: Midway through training the authors add an action-to-video pipeline for robotic manipulation using π0.5 to predict actions and Ctrl-World for video generation, and the resulting videos match ground truth more closely than direct I2V generation.
• Tool-use strategies generalize to unseen task combinations: On prompts combining a named character with a specialized action, the agent invokes both text and image retrieval without having seen that combination during training.

Methodology in Plain English

The agent operates in a reasoning–action–observation loop. Given a prompt, it reasons over the interaction history, picks a tool and its arguments, receives the tool's output as an observation, and repeats until it decides to return a video. The tools fall into three groups: augmentation (text search, image search, code-based simulation) that gathers guidance before generation; generation (text-to-video, image-to-video, multi-reference-to-video, motion-to-video); and verification (object detection, depth estimation) that inspects a candidate video so the agent can accept it or regenerate.

Each of the six task categories has a default workflow. For example, Procedural Knowledge runs text search then text-to-video; Single-Entity Identity runs image search then either multi-reference or image-to-video; Physics Simulation runs a simulation to produce motion conditions and passes them to a motion-conditioned generator; Multi-Shot generates a first segment, extracts its final frame, and uses that frame to condition each subsequent segment.

Training happens in two stages with Qwen3-VL-8B-Instruct as the base policy. Stage one is supervised fine-tuning on the 16K teacher trajectories using next-token prediction, masking failed action spans from the loss while keeping their error observations in context. Stage two initializes from that checkpoint and runs GRPO on the remaining 8K prompts, with only agent-emitted tokens contributing to the objective and tool responses masked as context. The reward is a hybrid of three parts weighted 0.1 for format validity, 0.5 for VLM-judged video quality under category rubrics, and 0.4 for task-appropriate tool use. After standard group-relative normalization, advantages are further normalized within each task category so that token-level signals from different categories are comparable. Rollouts that ultimately fail because of external tool errors are excluded from optimization.

Why This Matters

The work argues that visual quality and prompt fidelity are separable problems, and that agentic tool use with learned coordination is a way to attack the second. Rather than hand-building a pipeline per capability, one policy handles all six by learning when retrieval, simulation, detection, or depth feedback is needed. The demonstration that a trained agent gains from generation tools it never saw during training points to a modular division of labor: agent training and generator progress can advance independently.

Real-world applications the paper names or implies:

• Product demonstrations and educational content that require accurate depiction of specialized procedures.
• Sports analysis, where ordered sequences of actions must appear in the correct order and phases must not be omitted.
• Film pre-visualization, where multi-shot sequences need consistent temporal structure and visual continuity.
• Identity-sensitive content involving public landmarks, branded objects, game characters, or celebrities, where retrieval supplies visual references.
• Physically grounded scenes such as collisions, free fall, and celestial-body motion, where simulation supplies motion conditions.

Industry relevance: the common tool interface means compatible backends can be swapped in without redesigning the agent's workflows, which matters for production systems where generators are upgraded frequently. The coverage of both open-source and proprietary baselines, including CogVideoX-5B, Mochi-1, HunyuanVideo-13B, Wan2.1-T2V-14B, Hailuo 2.0, Kling 3.0, Seedance 1.0 Pro Fast, and Seedance 2.0 Fast, places the results in directly comparable terms.

Future Directions

• On-policy distillation to provide supervision on the agent's own trajectories rather than only on teacher-distilled ones, which the authors suggest could improve decisions across tasks.
• Stronger video-understanding models to supply more detailed feedback on generated content, so the agent can identify errors and make targeted corrections instead of coarse accept-or-regenerate decisions.
• Faster, higher-quality generation tools that are easier to deploy, reducing interaction cost and making iterative refinement more practical given the stated latency limitation.
• Broader task coverage and harder workflows, since the current limitation includes the six-task training setting and the coverage of verification feedback; jointly improving agent training, video feedback, and generation tools could support more complex creative tasks.

Target Audience

Researchers and engineers working on video generation, multimodal agents, and reinforcement learning with tool use will get the most from this paper, particularly those interested in how RL rewards can be designed for outputs that are videos rather than text answers. It is also relevant to practitioners building production video pipelines who need prompt fidelity for identity, physics, procedures, or multi-shot ordering, and to benchmark designers interested in category-specific VLM rubrics validated against human raters. Readers without background in diffusion models or policy-gradient methods will find the method sections dense, but the task taxonomy, workflow tables, and ablation results are readable on their own.

Authors’ abstract

Recent advances in video generative models have enabled high-fidelity, temporally coherent video generation. However, these models often struggle to satisfy prompts requiring specialized knowledge, specific identities, physical consistency, or ordered events. In this paper, we present VideoGen-Agent, a multimodal agent trained through multitask agentic reinforcement learning to use external tools for video generation. The agent coordinates augmentation, generation, and verification tools through multi-turn interactions, using the prompt and intermediate observations to guide its decisions. We train a shared policy on a category-balanced dataset spanning six tasks. Supervised fine-tuning on teacher-generated trajectories establishes tool-use behavior, which is then refined through reinforcement learning. A category-aware hybrid reward evaluates tool-call validity, task-appropriate tool use, and generated video quality. We further introduce VABench, a held-out benchmark of 600 prompts covering procedural knowledge, single- and multi-entity identity preservation, physical consistency, scene composition, and multi-shot temporal structure. On VABench, VideoGen-Agent improves over its base text-to-video generator by 19.1 points, from 56.5 to 75.6. Upgrading the generation tools further raises the score to 86.1 without additional agent training. Human raters prefer the upgraded configuration over the strongest standalone baseline in 84.3% of comparisons. These results support learning tool use across video-generation tasks and show that the trained agent can benefit from subsequent advances in generation tools.