AgenticGen: Reward-Guided Agentic Video Generation for Advertising

AI summary

Overview

Research area: Generative video models applied to advertising, combining agentic reasoning, reward modeling, and preference-based policy optimization.

Technical level: Advanced. The paper assumes familiarity with video diffusion/generative foundation models, reward modeling, and reinforcement-learning-style alignment techniques (DPO, GRPO).

Scope: The paper introduces AgenticGen, a reward-guided agentic framework that reframes advertising video generation as a two-stage, product-conditioned reasoning problem optimized against online business feedback.

What This Paper Is About

Advertising video generation is treated here not simply as a video synthesis task but as a reasoning problem conditioned on a product, where success is defined by online business outcomes rather than visual realism alone. Existing video foundation models can produce realistic clips from multimodal inputs, but they do not reason about how a product should be turned into an effective advertisement, nor do they learn from the business results of the ads they generate. The goal of AgenticGen is to close that loop by splitting ad creation into trainable reasoning stages and supervising them with rewards derived from both online performance and human quality standards.

Key Contributions

1. Reframing the task: The paper defines advertising video generation as a product-conditioned reasoning problem whose success is measured by online business metrics, rather than as pure video synthesis.

2. A two-stage agentic decomposition: The framework splits generation into strategy selection and draft generation, both trainable, which creates explicit optimization targets that online feedback can supervise.

3. A dual-reward design: AgenticGen learns a performance-based reward from accumulated online feedback plus a complementary rubric-based reward aligned with human quality standards, and uses both to supervise policy optimization.

4. A staged policy optimization pipeline with online validation: DPO first shifts the agentic policies toward online preferences, and GRPO then refines both stages using process and outcome rewards, with the resulting system evaluated offline and in online A/B tests within the TikTok advertising system.

Main Findings

• Two complementary rewards are used: a performance-based reward learned from accumulated online feedback, and a rubric-based reward aligned with human quality standards. The abstract does not specify how these rewards are weighted or combined.

• Offline experiments validate the components: the abstract states that offline experiments confirm the reward models and the successive policy optimization, but it reports no offline metrics, ablations, or dataset details.

• DPO and GRPO play distinct roles: DPO moves the agentic policies toward online preferences, after which GRPO further refines both stages using process and outcome rewards. The abstract does not report the incremental gain from each stage.

• Online A/B results in TikTok's advertising system: relative to the SFT baseline, AgenticGen after DPO and GRPO improves CTR by 2.72%, CVR by 2.63%, and Advv by 9.61%. These are the only quantitative results given.

• The gains are measured in business metrics, not just visual quality: the evaluation criterion is online advertising performance, consistent with the paper's framing of the task.

Methodology in Plain English

The researchers break ad video creation into two reasoning steps instead of asking one model to produce a finished video directly. First, the system selects a strategy — how the product should be presented and transformed into an advertisement. Second, it generates a draft video based on that strategy. Because these are separate stages, each can be scored and improved independently.

To score them, the authors build two reward signals. One learns from accumulated online business feedback, so it reflects what actually performed well with real audiences. The other is a rubric-based reward aligned with human quality standards, providing a complementary signal where online data alone might be sparse or noisy. These rewards then drive policy optimization in two phases: DPO to steer the policies toward online preferences, and GRPO to refine both stages with process and outcome rewards. The final system is tested offline and then in live A/B experiments inside TikTok's advertising system.

Why This Matters

Impact on research: The paper argues for treating generative video for commerce as a reasoning-and-optimization problem tied to measurable outcomes, and demonstrates a way to connect generative models to online business feedback loops — an alternative to optimizing solely for visual fidelity.

Real-world applications:

• Automated creation of product advertisements at scale for e-commerce and retail advertisers.
• Creative strategy selection that adapts to what has historically performed well for a given product category.
• Continuous improvement of ad creative from live campaign feedback rather than one-off human production.
• Quality control of machine-generated ads using rubric-based standards alongside performance signals.

Industry relevance: The evaluation is conducted in a live advertising system (TikTok), and the reported gains are in core business metrics — click-through rate, conversion rate, and advertising value — which are the metrics platforms and advertisers actually optimize. This positions the work at the intersection of generative AI and ad-tech, where small relative improvements translate into meaningful commercial impact.

Future Directions

• Per-stage attribution: The abstract reports combined DPO-plus-GRPO results but not the separate contributions of each optimization stage or of each reward component; isolating these would clarify where the gains come from.
• Generalization beyond one platform: Whether the learned rewards and agentic policies transfer across advertising systems, product categories, or regions is not addressed.
• Reward robustness and feedback latency: Reliance on accumulated online feedback raises questions about how quickly rewards adapt to changing trends and how noisy or delayed signals are handled.
• Broader task scope: Extending the two-stage decomposition to longer-form or interactive advertising formats, and to other product-conditioned generation tasks, is a natural extension the framing invites.

Target Audience

Researchers and practitioners working on generative video, multimodal generation, and RL-based alignment who are interested in applying these methods to commercial content; advertising and ad-tech engineers seeking to connect generative models to business metrics; and applied scientists at platforms that can run large-scale online A/B tests. Readers without background in preference optimization or reward modeling will find the methodology sections demanding, though the problem framing and results are accessible.

Authors’ abstract

Advertising video generation is not only a video synthesis task, but also a product-conditioned reasoning problem whose success is measured by online business metrics. Recent video foundation models can generate realistic clips from multimodal conditions, yet they do not optimize how a product should be transformed into an effective advertisement or how future generation should be improved from online business feedback. To close this loop, we propose AgenticGen, a reward-guided agentic framework that decomposes advertising video generation into two trainable reasoning stages, strategy selection and draft generation, thereby exposing optimization targets that online business feedback can supervise. AgenticGen learns a performance-based reward from accumulated online feedback and a complementary rubric-based reward aligned with human quality standards, then uses them to supervise policy optimization. DPO first moves the agentic policies toward online preferences, and GRPO further refines both stages with process and outcome rewards. Offline experiments validate the reward models and successive policy optimization. Online A/B experiments in the TikTok advertising system show that AgenticGen after DPO and GRPO improves CTR by 2.72%, CVR by 2.63%, and Advv by 9.61% over the SFT baseline.