Mind the Generative Details: Direct Localized Detail Preference Optimization for Video Diffusion Models

AI summary

Overview

• Research area: Post-training and preference alignment for text-to-video diffusion models (computer vision / generative AI).
• Technical level: Intermediate. The paper builds on Direct Preference Optimization (DPO) and latent video diffusion, but its central idea — comparing a real video with a locally corrupted copy of it — is conceptually simple.
• Scope: The paper proposes LocalDPO, a post-training framework that builds preference pairs by corrupting small spatio-temporal regions of real videos and optimizes the video diffusion model with a region-aware DPO loss.

What This Paper Is About

Aligning text-to-video diffusion models with human preferences usually means generating several videos for each prompt and having humans or a reward model rank them. That is slow, expensive, and the resulting scores are global, so they blur the specific local defects (flickering objects, distorted details) that viewers actually notice. The authors ask whether preference data can instead be built cheaply from real videos by deliberately degrading small regions of them, and whether the model can then be trained to prefer the clean regions over the degraded ones.

Key Contributions

1. LocalDPO preference-pair construction. A training-pair recipe that uses high-quality real videos as positive samples and locally corrupted versions of those same videos as negatives, requiring only a single inference per prompt and no external critic model or manual annotation.
2. Region-aware DPO loss. A mask-guided objective that restricts preference learning to the corrupted spatio-temporal regions, rather than applying preference gradients to the whole video.
3. Hybrid training objective. A combination of the region-aware DPO loss, standard (global) diffusion DPO loss, and a supervised fine-tuning (SFT) loss on real video latents, intended to keep local gains from damaging global video structure.
4. Empirical validation across three models. Experiments on CogVideoX-2B, CogVideoX-5B, and Wan2.1-1.3B, with automatic metrics on VBench and VideoJAM prompts plus a 20-participant user study, reported as outperforming the pretrained baseline, SFT, Vanilla DPO, and DenseDPO on most metrics.

Main Findings

• Visual quality improves most clearly. On VBench prompts, aesthetic quality for CogVideoX-2B rose from 0.6279 (baseline) to 0.6499 with LocalDPO, and imaging quality from 0.6589 to 0.7080. For CogVideoX-5B the same metrics moved from 0.6110 to 0.6274 and from 0.6631 to 0.7107; for Wan2.1-1.3B from 0.6363 to 0.6416 and from 0.6296 to 0.6412.
• VideoAlign overall scores improve. On VBench, VideoAlign Overall went from 7.7871 to 7.8568 (CogVideoX-2B), from 9.7191 to 10.2930 (CogVideoX-5B), and from 7.8300 to 7.9588 (Wan2.1-1.3B).
• VideoJAM prompt results follow the same pattern. Imaging quality for CogVideoX-5B rose from 0.6135 to 0.6727, and VideoAlign Overall from 7.5012 to 7.6424. For Wan2.1-1.3B, HPS-v2 rose from 0.6292 to 0.6667 and VideoAlign Overall from 7.1545 to 7.4849.
• Human preference metrics are mixed. HPS-v2 improved in all three VBench comparisons (for example 0.2655 to 0.2738 on CogVideoX-2B), but PickScore was essentially flat or slightly lower on CogVideoX-2B (21.50 to 21.46) and CogVideoX-5B (21.72 to 21.70). The paper states the method is superior on "the vast majority of metrics."
• Some VideoAlign sub-scores decline. Motion Quality (MQ) for CogVideoX-2B dropped from 0.7741 to 0.7173 on VBench and from 0.3849 to 0.3134 on VideoJAM; Image Reward on CogVideoX-2B VideoJAM dropped from 0.6407 to 0.7036 — actually it rose to 0.7036 — while Image Reward on CogVideoX-5B VideoJAM rose from 0.4805 to 0.5707. Not every individual metric moves in the same direction; the paper frames the gains as concentrated in visual quality and overall alignment.
• User study favorability. In a pairwise study with 20 participants on 50 VBench prompts, the authors report an average win rate of 88.86% across Visual Quality, Motion Quality, Text Alignment, and Overall Quality, with the largest gains in Visual Quality and Overall.
• Ablations on the losses. The comparison of the first three rows of the loss ablation shows the DPO and SFT terms giving only subtle boosts in visual quality and video alignment, while adding the region-aware DPO loss produces large improvements — the full combination reaches 0.6499 aesthetic quality and 0.7080 imaging quality versus 0.6279 and 0.6589 for the untrained baseline.
• Corruption matters more than using generated negatives. Training with real videos as positives and vanilla-DPO-generated videos as negatives reached 0.6285 aesthetic quality and 0.6577 imaging quality, below vanilla DPO's own 0.6304 and 0.6598. The authors attribute this to the large distribution gap between real and generated videos, which region-aware corruption narrows.
• Convergence. The paper reports that adding the region-aware DPO loss makes aesthetic and imaging quality rise faster during training and reach a higher ceiling, though the exact iteration-by-iteration values are only shown in a figure.
• Preference-pair cost. The paper states that constructing preference pairs with LocalDPO is faster in GPU time than vanilla DPO, illustrated in a figure; specific timing numbers are not given in the text.

Methodology in Plain English

Building the training data. The pipeline starts from a real, high-quality video paired with a text caption. The authors generate a random 3D mask: several Bézier curves are drawn and joined end-to-end into closed loops in the first frame, then randomly rotated and moved so the shapes extend across all frames, producing a spatio-temporal region to corrupt. The anchor points are sampled on a perturbed circle, with the number of primary vertices k drawn from 6 to 8, the radial corruption ratio ρ from [0.6, 0.8], the Bézier control-point scaling α from [0.2, 0.4], and the proposal region size h × w drawn from [H/3, H] and [W/3, W]. These masks mimic compact, organic occluders rather than rectangles.

Making the negative sample. The real video is encoded into a latent. A noise level α is sampled between α_l and α_h (set to 0.75 and 0.95 in the experiments), noise is added, and the pretrained model denoises it. After each denoising step, a fusion rule keeps the newly denoised latent only inside the mask and restores the re-noised original latent everywhere else: z_{t-1} = M ⊙ ẑ_{t-1} + (1 − M) ⊙ z_{t-1}^{orig}. This keeps masked and unmasked regions at the same noise level and prevents the whole video from drifting. The result is a video identical to the original outside the mask, with model-synthesized, degraded content inside it — the negative sample.

Training on the pairs. Instead of the standard DPO loss over the whole latent, the authors define a region-aware loss that measures the improvement of the model over a fixed reference model only inside the mask, rescaled by the ratio of total latent elements to mask elements (N_M / ||M||_1), and weighted by (1 + η(α)) where η(α) = (α − α_l) / (α_h − α_l). Heavier corruption therefore imposes a stronger penalty. The total objective is λ_RA-DPO · L_RA-DPO + λ_DPO · L_DPO + λ_SFT · L_SFT with weights 1.0, 1.0, and 0.1.

Experimental setup. Fine-tuning uses LoRA with rank 64 applied only to attention layers of the DiT, with the rest of the model frozen, for 540 iterations with batch size 128 and the AdamW optimizer. Generation at inference uses 50 DDIM steps with a classifier-free guidance scale of 6.0. The dataset consists of 63K high-quality clips curated from Pexels with content-tag filtering and human annotation, captioned with Qwen2.5-VL. Evaluation uses 165 VBench prompts from the aesthetic and imaging quality dimensions plus VideoJAM prompts, each expanded by Qwen2.5-VL, and compares against the pretrained baseline, SFT trained by LoRA on the same 63K dataset, Vanilla DPO (three videos per prompt ranked by a pretrained critic model), and DenseDPO (two videos per prompt ranked at frame level).

Why This Matters

• Impact on research. The paper argues that the bottleneck in video preference alignment is not the DPO objective itself but how preference pairs are constructed. By showing that a real video and its region-corrupted twin form a high-confidence pair, it suggests a cheaper and less ambiguous alternative to multi-sample ranking, and shifts attention from global quality scores to region-level supervision.
• Real-world applications:
  • Short-form and social video generation, where flickering objects and distorted details are the most visible failure modes.
  • Product and advertising video creation, where local detail fidelity of the featured item matters more than global video statistics.
  • Film, animation, and VFX previsualization, where artists need control over specific regions rather than whole clips.
  • Synthetic training or simulation footage, where the model must avoid localized artifacts in otherwise valid content.
• Industry relevance. The authors are affiliated with Alibaba Group (Taobao & Tmall Group) and Harbin Institute of Technology, and the method targets exactly the cost profile that matters industrially: one inference per prompt instead of

Authors’ abstract

Aligning text-to-video diffusion models with human preferences is crucial for generating high-quality videos. Existing Direct Preference Otimization (DPO) methods rely on multi-sample ranking and task-specific critic models, which is inefficient and often yields ambiguous global supervision. To address these limitations, we propose LocalDPO, a novel post-training framework that constructs localized preference pairs from real videos and optimizes alignment at the spatio-temporal region level. We design an automated pipeline to efficiently collect preference pair data that generates preference pairs with a single inference per prompt, eliminating the need for external critic models or manual annotation. Specifically, we treat high-quality real videos as positive samples and generate corresponding negatives by locally corrupting them with random spatio-temporal masks and restoring only the masked regions using the frozen base model. During training, we introduce a region-aware DPO loss that restricts preference learning to corrupted areas for rapid convergence. Experiments on Wan2.1 and CogVideoX demonstrate that LocalDPO consistently improves video fidelity, temporal coherence and human preference scores over other post-training approaches, establishing a more efficient and fine-grained paradigm for video generator alignment.The code is available at https://github.com/1170300714/Local-DPO.