From Preferences to Prejudice: The Role of Alignment Tuning in Shaping Social Bias in Video Diffusion Models

AI summary

Overview

Research area: Social bias and fairness in generative AI, specifically text-to-video diffusion models and the preference-alignment pipelines (reward models trained on human preferences) used to post-train them.

Technical level: Intermediate. The paper assumes familiarity with diffusion models, reward-weighted fine-tuning / RLHF-style post-training, and standard fairness metrics, but explains its own metrics in full detail.

Scope: A single paper introducing the VideoBiasEval diagnostic framework and reporting the first end-to-end trace of how gender and ethnicity bias moves from human preference datasets, into image reward models, and finally into alignment-tuned video diffusion models.

What This Paper Is About

Video diffusion models are often improved by "alignment tuning," where a reward model trained on human preferences guides further optimization. The problem is that human preferences are not neutral, so alignment may quietly change who appears in generated videos, how often, and in what roles. The paper builds a framework to measure this and traces bias from preference datasets through reward models to the final video generators.

Key Contributions

1. VideoBiasEval, a diagnostic framework for social representation in video generation that uses event-based prompting (an actor performing a verb in a context) to separate semantic content from actor attributes, plus multi-granular metrics for ethnicity bias, gender bias conditioned on ethnicity, distributional shifts across model variants, and temporal persistence of bias.

2. The first end-to-end analysis connecting biases in human preference datasets (HPDv2, Pick-a-Pic), their amplification in image reward models (HPSv2.0, HPSv2.1, PickScore), and their propagation into alignment-tuned video diffusion models.

3. Evidence that alignment tuning reshapes social composition of generated content — improving perceptual quality and temporal coherence while in some cases reinforcing or stabilizing existing social biases.

4. Guidance for mitigation via controllable preference modeling, showing how targeted data composition and counter-biased reward design can steer video diffusion models toward more equitable behavior.

Main Findings

• Human preference datasets are not demographically neutral. After filtering to valid man–woman pairs, 26 of 42 verbs in HPDv2 met the criteria, and 69.23% (18/26) showed a preference for men. In Pick-a-Pic, 18 of 42 verbs qualified, and 61.11% (11/18) showed a preference for women. Both datasets strongly favored White representation — 43.34% in HPDv2 and 40.08% in Pick-a-Pic — followed by East Asian and Indian representations.

• Ethnic overrepresentation is severe under person-only prompts. ModelScope showed RDS_White = 0.769 with SDI = 0.0538. The paper notes that an RDS_e of 0.769 means a group appears in roughly 91.2% of generated videos (0.769 + 0.143), more than six times the expected frequency under a uniform 1/7 ≈ 14.3% reference.

• Alignment tuning amplified gender skew. InstructVideo's average PBS_G (Proportion Bias Score for Gender) rose by 0.0480 over ModelScope (from 0.4815 to 0.5295), and T2V-Turbo-V1 rose by 0.0725 over VideoCrafter-2 (from 0.7581 to 0.8306), indicating a shift toward male portrayals after preference-based fine-tuning.

• Ethnic concentration and diversity showed a trade-off. InstructVideo increased White RDS from 0.7690 to 0.7833 and lowered SDI from 0.0538 to 0.0267. T2V-Turbo-V1 reduced White RDS from 0.6905 to 0.6381 but also lowered SDI from 0.1252 to 0.1119 — less White dominance, but less overall diversity.

• Some models produced zero recognizable instances of certain ethnic groups. Table 2 marks "—" for cases where a model generated zero recognizable instances of an ethnicity under the person-only condition, which the authors describe as a severe lack of diversity that prevents metric computation for those subgroups.

• Bias becomes temporally stable after alignment. The new Temporal Attribute Stability (TAS) metric measures the percentage of frames whose classified attributes match the video's final majority-voted label. The authors report that alignment tuning improves consistency of social attribute portrayals over time, meaning biased representations become more persistent and visually stable.

• Able automated evaluators were validated against humans. Across 400 generated videos annotated by three independent annotators, the three-VLM ensemble achieved Pearson correlations of 0.89 (gender) and 0.73 (ethnicity). Cohen's Kappa reached 0.91 (gender) and 0.78 (ethnicity), with Fleiss' Kappa of 0.92 and 0.82 respectively.

• Evaluator disagreement did not change conclusions. On a high-agreement subset, consensus rates were 99.0% for gender and 83.3% for ethnicity, and core metrics stayed statistically stable (PBS_G: 0.846 → 0.863, p = 0.68; SDI: 0.745 → 0.761), described as a non-significant difference.

Methodology in Plain English

The researchers define an "event" as a tuple of an actor, a verb, and a context. They curate 42 verbs previously documented as statistically associated with particular genders or ethnic groups, and 4 gender categories (man, woman, "person," and non-binary person) and 7 ethnic groups (White, Black, Indian, East Asian, Southeast Asian, Middle Eastern, Latino).

Prompts are generated deterministically from a template — "A/An [actor] is [verb]-ing [context]" — rather than written by an LLM at runtime. Two conditions are used: a person-only condition (42 prompts, actor fixed to "person") and an ethnicity+person condition (294 prompts, formed by permuting the same 42 verbs across seven ethnic descriptors). Videos are generated 10 times per prompt per model with different random seeds and averaged to reduce sampling variance.

Because demographic attributes are never stated during generation aside from these prompt templates, all demographic analysis happens after the fact. For each generated video, 16 frames are uniformly sampled and three open-source vision-language models — Qwen2-VL-7B, Qwen2.5-VL-7B, and InternVL2.5-8B — classify gender and ethnicity in each frame. Predictions are majority-voted within each model to get video-level labels and then ensembled across models, because the authors found single VLMs often unstable or hallucinated attributes.

Bias is then quantified with four metric families: the Proportion Bias Score for Gender (PBS_G), which ranges from −1 to 1 and measures the man-minus-woman balance; the Representation Deviation Score for Ethnicity (RDS_e), which measures each group's deviation from a uniform 1/|E| reference; Simpson's Diversity Index (SDI) for overall balance; and Temporal Attribute Stability (TAS) for within-video consistency. Bias shift (Δ) values compare aligned models against their unaligned baselines. Four video diffusion models are evaluated in two aligned/unaligned pairs: ModelScope versus InstructVideo (aligned with HPSv2.0), and VideoCrafter-2 versus T2V-Turbo-V1 (aligned with HPSv2.1, InternVid2-S2, and ViCLIP). For the dataset and reward model analysis, the authors extract attributes from captions with GPT-4o-mini and from images with the same three VLMs.

Why This Matters

Impact on research. The paper reframes alignment tuning as a bias-shaping intervention rather than a purely quality-improving one. It shows that reward models trained on human preferences inherit and amplify demographic skew, and that the resulting bias is not just present but temporally stable — which makes it harder to dismiss as random sampling noise. It also supplies a reproducible benchmark and metric suite where prior text-to-video evaluations such as VBench, EvalCrafter, and T2V-CompBench focused on fidelity and consistency rather than who is depicted.

Real-world applications:

• Creative media generation, where skewed representation in defaults can shape the visual culture of advertising, film previsualization, and stock content.
• Educational content creation, where biased depictions of who performs which roles can reinforce stereotypes for learners.
• Professional simulation and training video generation, where realistic demographic distribution matters for scenario fidelity.
• Preference dataset curation and reward model auditing, where the findings give practitioners concrete signals (gender preference rates, ethnicity proportions) for checking their own data pipelines before release.

Industry relevance. Alignment tuning with image reward models is described as the de facto and state-of-the-art approach for post-training video diffusion models. That matters because any company adopting this standard pipeline inherits any bias in the reward data. The paper's Section 7 explores whether controllably constructed reward datasets can deliberately steer models toward more equitable outputs, which is directly actionable for teams building or fine-tuning these systems.

Future Directions

• Scaling beyond the bounded attribute space. The authors deliberately restrict analysis to 4 gender categories, 7 ethnic groups, and 42 verbs for control and interpretability, and explicitly call expanding to open-ended or intersectional attributes a promising next step.

• Larger-scale human annotation. The paper validates its VLM ensemble on 400 videos with three annotators, acknowledging that larger-scale human annotation would be ideal but is constrained by labor costs. The gap between the human–model Pearson correlation of 0.89 for gender and 0.73 for ethnicity also points to frames where human holistic judgment and frame-level VLM precision diverge.

• Counter-biased reward model design. The authors outline how targeted data composition and counter-biased reward modeling can serve as mitigation strategies, but the paper reports these as guidance rather than a fully resolved solution.

• Broader model and data coverage. The end-to-end trace covers two aligned/unaligned model pairs and notes analyses of training data effects (WebVid-10M) and statistical tests in the appendix, leaving room for extension across more architectures, more reward models, and more preference datasets.

Target Audience

Researchers and practitioners working on generative video, preference-based alignment, and AI fairness. It is most useful for teams building or fine-tuning text-to-video systems with reward models, for dataset curators assembling human preference data, and for fairness researchers who need a concrete, reproducible protocol for auditing demographic representation in temporally extended generated media. Readers without prior exposure to diffusion models or RLHF-style post-training will find the metric definitions accessible but the pipeline context demanding.

Authors’ abstract

Recent advances in video diffusion models have significantly enhanced text-to-video generation, particularly through alignment tuning using reward models trained on human preferences. While these methods improve visual quality, they can unintentionally encode and amplify social biases. To systematically trace how such biases evolve throughout the alignment pipeline, we introduce VideoBiasEval, a comprehensive diagnostic framework for evaluating social representation in video generation. Grounded in established social bias taxonomies, VideoBiasEval employs an event-based prompting strategy to disentangle semantic content (actions and contexts) from actor attributes (gender and ethnicity). It further introduces multi-granular metrics to evaluate (1) overall ethnicity bias, (2) gender bias conditioned on ethnicity, (3) distributional shifts in social attributes across model variants, and (4) the temporal persistence of bias within videos. Using this framework, we conduct the first end-to-end analysis connecting biases in human preference datasets, their amplification in reward models, and their propagation through alignment-tuned video diffusion models. Our results reveal that alignment tuning not only strengthens representational biases but also makes them temporally stable, producing smoother yet more stereotyped portrayals. These findings highlight the need for bias-aware evaluation and mitigation throughout the alignment process to ensure fair and socially responsible video generation.