VMDT: Decoding the Trustworthiness of Video Foundation Models

AI summary

Overview

• Research area: Trustworthiness evaluation of video foundation models (AI safety, multimodal evaluation, computer vision)
• Technical level: Intermediate — accessible to readers familiar with multimodal models and benchmarking, though the full paper assumes some background in evaluation metrics
• Scope: A single unified benchmark (VMDT) that measures five trustworthiness dimensions across 7 text-to-video (T2V) and 19 video-to-text (V2T) models, with a corresponding analysis of how these properties scale with model size.

What This Paper Is About

Trustworthiness benchmarks exist for language and image models, but the video modality has no comparable standard, despite video-specific risks like temporal harm and physical harm that cannot be detected in a single frame. The authors build VMDT (Video-Modal DecodingTrust), the first unified platform for probing T2V and V2T models across safety, hallucination, fairness, privacy, and adversarial robustness. The goal is not only to score models but to identify which trustworthiness properties improve or degrade as models get larger.

Key Contributions

1. A unified, five-dimension trustworthiness benchmark for video. VMDT provides curated datasets and metrics for safety, hallucination, fairness, privacy, and adversarial robustness under one framework, covering both generation (T2V) and understanding (V2T) directions.
2. Video-specific risk taxonomies and scenarios. The safety dataset introduces risks unique to video, such as temporal risks (harm emerges only across a sequence of frames) and physical harm (e.g., strobing that could trigger photosensitive epilepsy), plus "transformed" prompts that disguise harmful intent in benign wording.
3. The "overkill fairness" metric. Beyond standard stereotype and decision-making bias scores, the paper quantifies when models sacrifice factual or historical accuracy in pursuit of diversity — for example, generating historically inaccurate demographics.
4. Scale analysis across 26 models. By evaluating models of varying sizes within the same families, the authors map how each trustworthiness dimension correlates with model scale, revealing that some risks worsen with size while others improve.

Main Findings

• Safety does not scale with model size. Unlike every other dimension, safety shows no correlation with parameter count, implying that current safety levels are governed by alignment choices rather than capability. All evaluated open-source T2V models had a 100% bypass rate on harmful prompts, and even the best closed-source model (Nova Reel) only reduced this to 37%.
• Open-source T2V models lack refusal mechanisms entirely. Every open-source T2V model failed to reject harmful queries. Closed-source models handled video-specific hazards (temporal risks, physical harm) poorly, and V2T closed-source models remained vulnerable to fraud, deception, and illegal activity categories.
• Hallucination improves with scale in V2T but open-source T2V lags badly. All open-source T2V models fell below 50% average accuracy, while Luma (67.6%) and Pika (63.0%) led. For V2T, the open-source InternVL2.5-78B outperformed all closed-source models, with performance rising monotonically with size within families — though the best models still only reach roughly 65% accuracy.
• Counting and spatial understanding are consistently the hardest tasks. Average accuracy stayed under 50% across models for both, echoing known weaknesses in text-to-image models. OCR generation in video was also near-failure for most T2V models.
• Unfairness and privacy risk increase with model size in V2T. Larger V2T models showed stronger gender and age stereotypes and more precise geolocation inference from driving videos (Pearson correlation 0.544, p = 0.016). GPT-4o and Claude-3.5-Sonnet posed the highest location-inference privacy risk.
• T2V models are biased but less prone to overkill fairness than T2I models. T2V models overrepresented male, White, and younger subjects more strongly than text-to-image models, yet showed lower overkill fairness — suggesting a trade-off between the two failure modes.
• Both model types are vulnerable to adversarial inputs, with different weak spots. Gradient-based prompt attacks dropped T2V performance by 5.0% on average (11.4% on action recognition). V2T models were least robust on counting tasks, with drops up to 18.1%. Larger V2T models were modestly more robust (p = 0.034).
• Data memorization is weak in T2V, with one exception. Generated videos were far from training data at the pixel level (cosine similarity around 0.24 vs. around 0.7 for T2I models), but VideoCrafter2 occasionally reproduced copyrighted watermarks, indicating partial memorization.
• No model is trustworthy across the board. The best T2V model (Luma) averaged 70.1 out of 100 and the worst (CogVideoX-5B) 55.7; the best V2T model (InternVL2.5-78B) scored 72.7 and the worst (Qwen2.5-VL-3B-Instruct) 65.3. No model topped all five dimensions simultaneously.

Methodology in Plain English

The authors assembled five separate evaluation datasets, one per trustworthiness dimension, drawing on established sources and adapting them for video. Safety prompts and videos came from industry policy taxonomies and prior multimodal safety benchmarks; hallucination tests reused VATEX, CLEVRER, and Neptune material, organized into scenarios such as misleading context, counterfactual prompts, and temporal transitions; fairness prompts were adapted from a text-to-image benchmark plus video datasets for V2T.

Evaluation relied on automated judges. For T2V safety, ten frames per generated video were scored by GPT-4o, which agreed with human annotators 86% of the time. V2T outputs were judged similarly (88% agreement). Hallucination accuracy was measured with Qwen2.5-VL-72B-Instruct for T2V and keyword matching for V2T multiple-choice questions. Fairness was decomposed into three numeric metrics — stereotype bias, decision-making bias, and overkill fairness — each normalized so that zero is ideal. Privacy was tested two ways: pixel-level distance between generated and training videos for T2V, and geolocation inference accuracy from driving footage for V2T. Adversarial robustness was tested by attacking surrogate models with several optimization algorithms and measuring the accuracy drop between clean and adversarial inputs.

Why This Matters

Research impact. VMDT fills a clear gap: prior trustworthiness work covered text and images, leaving video unmeasured. Its finding that safety is decoupled from scale challenges the assumption that capability gains bring safety gains, and it gives the community a shared yardstick for tracking whether future video models improve.

Real-world applications:

• Video generation platforms can use the safety and overkill-fairness tests to audit models before deployment, particularly for strobing effects that pose genuine health risks.
• Content moderation pipelines can adopt the V2T safety benchmark to check whether video-understanding systems resist or amplify harmful intent in the videos they describe.
• Privacy compliance and regulation — the geolocation inference results are directly relevant to regulators assessing whether video models can re-identify locations from everyday footage such as dashcams.
• Media and advertising production can use the fairness metrics to detect systematic demographic skew in generated casting and representation.

Industry relevance. The evaluation spans both open-source and commercial models from multiple providers, making the open-versus-closed safety gap actionable for procurement and release decisions. Because the benchmark is public and reproducible, it supports third-party auditing rather than self-reported safety claims.

Future Directions

• Why safety does not scale. The paper shows safety is orthogonal to model size but does not explain what governs it; isolating the alignment techniques or data properties that drive safety is an open problem.
• Mitigating scale-linked harms. Fairness and privacy risks worsen with parameter count in V2T models. Training or post-hoc interventions that reverse this trend are unexplored.
• Multilingual and cross-cultural extension. The current fairness taxonomy covers gender, race, and age within a largely Western framing; broader demographic and cultural coverage would test whether the identified biases generalize.
• Closing the video-specific safety gap. Temporal and physical-harm risks defeated essentially all models. Building refusal and detection mechanisms that operate across frames, rather than per-frame, remains unresolved.

Target Audience

AI safety and alignment researchers, benchmark designers, and model evaluation engineers will get the most from this paper, along with product and policy teams at organizations deploying or procuring video generation and video understanding systems. Regulators and auditors concerned with multimodal privacy and fairness will also find the metrics and taxonomy directly useful, while general machine learning readers can treat the cross-perspective findings as a concise status report on where video foundation models currently fail.

Authors’ abstract

As foundation models become more sophisticated, ensuring their trustworthiness becomes increasingly critical; yet, unlike text and image, the video modality still lacks comprehensive trustworthiness benchmarks. We introduce VMDT (Video-Modal DecodingTrust), the first unified platform for evaluating text-to-video (T2V) and video-to-text (V2T) models across five key trustworthiness dimensions: safety, hallucination, fairness, privacy, and adversarial robustness. Through our extensive evaluation of 7 T2V models and 19 V2T models using VMDT, we uncover several significant insights. For instance, all open-source T2V models evaluated fail to recognize harmful queries and often generate harmful videos, while exhibiting higher levels of unfairness compared to image modality models. In V2T models, unfairness and privacy risks rise with scale, whereas hallucination and adversarial robustness improve -- though overall performance remains low. Uniquely, safety shows no correlation with model size, implying that factors other than scale govern current safety levels. Our findings highlight the urgent need for developing more robust and trustworthy video foundation models, and VMDT provides a systematic framework for measuring and tracking progress toward this goal. The code is available at https://sunblaze-ucb.github.io/VMDT-page/.