TEAR: Temporal-aware Automated Red-teaming for Text-to-Video Models

AI summary

Overview

Research area: Computer Vision / AI safety — specifically adversarial safety evaluation (red-teaming) of Text-to-Video (T2V) generative models.

Technical level: Advanced. The paper assumes familiarity with diffusion-based video generation, reinforcement learning from preference feedback (PPO), and reward modeling, though its central idea is explained here in plain language.

Scope in one sentence: The paper introduces TEAR, an automated red-teaming framework that trains a prompt generator to exploit the temporal structure of video — sequences of individually harmless moments that aggregate into a policy-violating video — and shows it produces attack success rates above 80% on open-source and commercial T2V systems, versus a prior best of 57%.

What This Paper Is About

T2V models such as Veo, Hailuo, and Wan can turn a text prompt into a coherent, realistic video, and with that capability comes the risk of producing harmful content. Existing safety evaluation tools were built for static images and text, so they treat a video as just a stack of independent frames and miss dangers that only appear when events unfold over time. TEAR's goal is to automatically discover "textually safe" prompts — prompts that any text filter would approve — that nevertheless cause a T2V model to render an unsafe video once the events play out in sequence.

Key Contributions

1. TEAR framework: The first automated red-teaming framework aimed specifically at temporal vulnerabilities in T2V models, designed to find safety risks that emerge from dynamic event sequencing rather than from any single frame or word.
2. Temporal-aware generator optimization: A test generator trained in two stages — initial generator training, then temporal-aware online preference learning with a prompt-level reward and a temporal consistency reward — to steer a large language model toward prompts that are textually innocuous but temporally harmful.
3. Comprehensive evaluation: Experiments on five leading T2V models (open-source Wan 2.2-14B and Hunyuan-Video; commercial Veo-3.1, MiniMax Hailuo-2.3, and Luma Ray-2) against four state-of-the-art baselines across six unsafe categories, plus three safety filters.
4. Exposure of commercial API safety failures: The paper shows that prompts with near-perfect pass rates through commercial API and NSFW filters still yield high attack success rates, demonstrating that current safety filters are insufficient for dynamically unsafe cases.

Main Findings

• High attack success on open-source models: On Hunyuan-Video, TEAR reaches an 82.3% ASR (321 of 390 seeded attempts), well above FLIRT's 57.2% (223). On Wan 2.2, TEAR reaches 80.5% ASR (314) against FLIRT's 56.4% (220).
• Text-only baselines fail: The Naive approach — ordinary video generation prompts — succeeds only 2.3% to 2.6% of the time. T2VSafetyBench scored 40.8% on Hunyuan-Video and 37.2% on Wan 2.2; UVD scored 29.0% and 31.0%; ART scored 52.6% and 49.7%.
• Strong across categories: On Hunyuan-Video, TEAR's successful cases by category were Violence 61, Gore 55, Self-harm 58, Pornography 29, Illegal Activity 57, and Disturbing Content 61. On Wan 2.2 the counts were 57, 63, 56, 27, 53, and 58.
• Prompts stay textually safe: Prompt Successful Pass rates for TEAR were 96.7% (Toxicity Detector), 96.5% (NSFW), and 95.6% (Llama Guard) on Hunyuan-Video, and 94.3%, 97.4%, and 94.9% on Wan 2.2. In contrast, FLIRT passed at roughly 48% to 53%.
• Works without seed prompts: In a seed-free setting, TEAR achieved 79.2% ASR on Hunyuan-Video and 76.9% on Wan 2.2, versus FLIRT at 55.1% and 53.8%, while maintaining NSFW pass rates of 96.2% and 97.0%.
• Refinement rounds help: ASR begins at 57%–71% in the direct setting, rises sharply over the first three rounds, and reaches 83%–95% after 8 rounds. The NSFW Filter Pass Rate starts at 79%–86% and surpasses 95%.
• Commercial services are vulnerable: Prompts achieve near-perfect API-level and NSFW Filter Pass Rates approaching 98.0%, yet produce ASRs generally at or above 85.0% for most categories such as Violence. Pornography registers the lowest ASR, falling below 80.0%.
• Prompts transfer across models: Across all 20 source-target combinations of five T2V models, the average transfer ASR is 76.4%, with most values between 70% and 82%. Prompts optimized for Wan 2.2 hit 80.8% on Veo-3.1, and prompts from Ray-2 peaked at 82.6% on Hailuo-2.3.
• Prompts are diverse: Using 1-AvgSelfBLEU and 1-Cossim (where higher means more diverse), scores ranged from about 0.71 (Self-harm) to 0.76 (Illegal Activity), and 0.69 (Disturbing Content) to 0.73 (Pornography).
• Generation settings matter: ASR rises with inference steps and plateaus around 50 steps. ASR varies with the CFG scale and peaks at a moderate value, suggesting that clearer visual output helps the video judge and refine model work.

Methodology in Plain English

The researchers reframe red-teaming as a search problem and split it into three moving parts: a temporal-aware test generator, a refine model, and the target T2V model.

First, they build a dataset of harmful seed prompts (390 in total, 65 per category across six harmful categories) and use a rule-based rewriting process to convert each harmful instruction into a chronological sequence of harmless-sounding events. Three rewriting rules are used: Temporal Deconstruction (break the harmful directive into discrete static event descriptions), Sequential Enforcement (add explicit temporal connectives such as "First" or "After two seconds"), and Temporal-Space Synthesis (harm emerges only from the composition of those events, not from any one of them). The rewritten prompts are filtered so that a text judge calls them safe while the corresponding video is judged unsafe.

Second, a base Llama-3 model is fine-tuned with LoRA on this dataset using a next-token negative log-likelihood loss, giving an initialized generator. This generator is then improved through online preference learning framed as a Markov Decision Process and optimized with the PPO paradigm, using two rewards: a prompt-level reward that combines a hate speech classifier's confidence with a cosine similarity score that keeps prompts stylistically close to the rule-based temporal prototypes, and a temporal consistency reward that checks whether the generated video matches the original harmful semantics (global consistency) and whether the video is internally coherent (inner consistency). A Kullback-Leibler penalty between the policy model and the initial generator prevents over-optimization.

Third, a refine model (Qwen-3-VL, used with few-shot in-context learning) closes the loop. After the target model renders a video from the generated prompt, a textual judgment system and a video judgment system produce structured feedback — a quantitative score, an explanation, and an actionable suggestion — and the refine model revises the prompt accordingly, repeating until the red-teaming objective is met. The authors frame the threat model as a T2V developer auditing their own model before public release, using auxiliary large language and vision-language models as automated evaluators.

Why This Matters

Impact on research. These results suggest that the failure mode being exploited — harm that exists only in the temporal composition of events — is shared across T2V architectures, since prompts optimized on one model transferred to others at rates mostly above 70%. That implies video safety cannot be solved by frame-level or prompt-level filtering alone, and it gives the safety community a concrete benchmark and a tool for studying temporal alignment.

Real-world applications (implied by the paper's framing):

• Pre-release safety auditing of T2V models by developers, using TEAR to surface temporal flaws before deployment.
• Hardening safety filters for commercial T2V API services, which the paper shows admit prompts at near-98% pass rates that still generate harmful video.
• Building alignment training data, since the discovered problematic prompts can be used to fine-tune models toward safer behavior.
• Evaluating video content moderation pipelines, which currently inherit frame-oriented assumptions.

Industry relevance. Commercial T2V services from Google, MiniMax, and Luma are directly affected: TEAR produced high ASRs against Veo-3.1, Hailuo-2.3, and Ray-2 despite their filters. The paper reports that ASR on Ray-2 was slightly lower, which the authors hypothesize is due to lower video-prompt consistency as shown by VBench, rather than stronger safety. For any company shipping video generation, this is evidence that text filters are an inadequate safety layer.

Future Directions

• Closing the temporal alignment gap. Since transferability suggests a shared weakness, a natural next step is designing defenses that reason over event sequences rather than frames or prompts in isolation.
• Improving defenses on lower-consistency models. The authors hypothesize that Ray-2's lower ASR stems from weaker video-prompt consistency. Whether that can be deliberately engineered as a safety property, or whether it is incidental, is unresolved.
• Scaling the evaluation breadth. The work covers five T2V models, three safety filters, six harmful categories, and 390 seed prompts; extending to more models, categories, and longer videos would test how general the temporal attack is.
• Understanding the refine loop's limits. ASR growth moderates after about three refinement rounds and plateaus after 8 rounds. What caps that improvement, and whether stronger refine models or feedback signals could push it further, remains open.

Target Audience

This paper is most useful to AI safety researchers studying generative model robustness, T2V model developers and platform engineers responsible for content safety and pre-release auditing, red-teaming and alignment practitioners who need automated attack tooling, and policy or trust-and-safety teams evaluating whether published video generation APIs meet their safety claims. Readers who want a first look at video-specific safety failures will benefit even without deep reinforcement learning background, though the methodology sections assume it.

Note: this paper includes a warning that it contains model outputs which are offensive in nature.

Authors’ abstract

Text-to-Video (T2V) models are capable of synthesizing high-quality, temporally coherent dynamic video content, but the diverse generation also inherently introduces critical safety challenges. Existing safety evaluation methods,which focus on static image and text generation, are insufficient to capture the complex temporal dynamics in video generation. To address this, we propose a TEmporal-aware Automated Red-teaming framework, named TEAR, an automated framework designed to uncover safety risks specifically linked to the dynamic temporal sequencing of T2V models. TEAR employs a temporal-aware test generator optimized via a two-stage approach: initial generator training and temporal-aware online preference learning, to craft textually innocuous prompts that exploit temporal dynamics to elicit policy-violating video output. And a refine model is adopted to improve the prompt stealthiness and adversarial effectiveness cyclically. Extensive experimental evaluation demonstrates the effectiveness of TEAR across open-source and commercial T2V systems with over 80% attack success rate, a significant boost from prior best result of 57%.