ID-Crafter: VLM-Grounded Online RL for Compositional Multi-Subject Video Generation

AI summary

Overview

Research area: Computer vision, specifically multi-subject video generation using diffusion transformers, vision-language models (VLMs), and reinforcement learning (RL).

Technical level: Intermediate (assumes familiarity with diffusion models, attention mechanisms, and basic RL concepts).

Scope: A single framework — ID-Crafter — that generates temporally coherent videos preserving the identities of multiple reference subjects, backed by quantitative benchmarks and ablations.

What This Paper Is About

Existing video generation models struggle when asked to combine several distinct subjects (say, two people and a specific object) drawn from separate reference images. Injecting multiple identity signals into one diffusion model causes semantic conflicts — identities bleed into each other, prompts get ignored, and subjects degrade over time. ID-Crafter's goal is to preserve every subject's identity with high fidelity while still producing natural motion and correctly following the text prompt.

Key Contributions

1. A hierarchical identity-preserving attention mechanism that processes subject information in three cascaded stages: intra-subject (refining each subject's own features), gated inter-subject (modeling relationships between subjects while blocking identity leakage), and cross-modal (fusing subjects with text and video tokens).

2. A VLM as a semantic reasoning engine. The paper claims to be the first open-source work to place a pretrained vision-language model (Qwen2.5-VL-7B) as the core reasoning component inside the Wan-Video architecture, treating it as a dynamic semantic guide rather than a static encoder.

3. An online reinforcement learning post-training stage. The authors apply GRPO (groupwise policy optimization) with a composite reward combining identity fidelity and video quality — the first application of online RL to multi-subject video generation, reportedly stable enough to work on this complex architecture.

4. A new training dataset and evaluation protocol. A curation pipeline combining real subject-video pairs, synthetic cross-subject compositions generated by image editing models, and professionally shot footage, paired with the OpenS2V-Nexus benchmark for held-out evaluation.

Main Findings

• State-of-the-art across the board: The 14B model reaches a 57.05% total score, surpassing all open-source baselines (VACE-14B at 52.87%, Phantom-14B at 52.32%) and even the proprietary Kling 1.6 (54.46%).

• Small model, large gains: The 1.3B variant with online RL achieves 66.10% FaceSim, beating much larger baselines. The 14B model reaches 40.34% Motion score versus Phantom-14B's 33.42%.

• Online RL beats offline DPO: FaceSim improves from 58.12% (SFT baseline) to 62.35% (offline DPO) to 66.10% (online GRPO/ID-Crafter), with the same trend in aesthetics and Q-Align perceptual quality.

• Hierarchical attention is critical: Removing it drops FaceSim by 11.7%. Within the hierarchy, the cross-modal stage matters most — ablating it costs 7.2 points of FaceSim vs. 2.9 and 2.6 for the intra- and inter-subject stages.

• VLM encoder drives prompt alignment: Swapping the dual T5+VLM encoder for T5 alone reduces Q-Align by 18.2%, the largest single-component degradation in the ablation.

• Curated data reduces "copy-paste" artifacts: Training without the curated dataset drops total score by 10.2% and produces visibly worse subject integration.

• Reward hacking is real and measurable: Removing the naturalness reward raises FaceSim to 69.30% but lowers Q-Align to 0.361 — evidence that the model over-optimizes visible metrics at the expense of physical plausibility.

• Zero-shot video editing works: The architecture can replace subjects, insert new ones, or alter backgrounds in existing videos while maintaining temporal consistency.

Methodology in Plain English

The system starts with a latent video diffusion transformer (Wan-Video 1.3B, trained with Rectified Flow). Reference images of each subject are encoded into token sequences.

Instead of dumping all subject tokens and text into the model at once, the authors insert a three-stage attention block into the first 20 layers of the transformer. Stage one lets each subject's tokens attend to themselves, capturing fine detail. Stage two uses a gated cross-attention where each subject queries the others, but a learned gate controls how much information passes — this is the key trick for preventing identity bleed. Stage three cross-attends video latents to the combined subject and text tokens.

For the text side, a VLM processes the prompt and the reference images together, producing semantically rich tokens that understand which subject does what. This beats a plain text encoder because the VLM can reason about relationships ("the man in the red shirt holds the dog").

After this supervised training, the model undergoes online RL. For each input, it samples a group of videos, scores each with a composite reward (60% identity fidelity, 40% quality), and updates the policy by comparing videos within the group — no separate value network needed. The reward blends face similarity (ArcFace), holistic subject consistency, aesthetic score, and a VLM-judged naturalness score. A contrastive mechanism stabilizes the hierarchical attention during this phase.

Why This Matters

Research impact: The paper demonstrates that online RL can be applied stably to complex multi-subject video diffusion architectures, and that a VLM can serve as more than an encoder — as an active semantic guide. Both are templates other generative systems could adopt.

Real-world applications:

• Personalized video creation where users place themselves and others into generated scenes
• Advertising and product placement with specific branded items rendered consistently
• Virtual storytelling and animation with recurring characters
• Post-production video editing — swapping actors or backgrounds while preserving identity

Industry relevance: The improvements over proprietary models like Kling and Pika on aggregate score suggest that open-source pipelines with the right architectural choices can compete commercially. The one-minute generation time for a 480p video on a 1.3B model is practical for near-real-time creative workflows.

Future Directions

• Complex interactions and fine-grained dynamics remain weak. The authors acknowledge the model still struggles with nuanced subject-to-subject interactions and subtle motion.

• Physics-aware priors. Injecting physical constraints could reduce the need for the naturalness reward and eliminate reward hacking at its source.

• Bias mitigation in pretrained components. The VLM, VAE, and image encoder all carry biases that propagate into generation; the paper flags this as unresolved.

• Attribute- and action-level controllability. Going beyond identity to let users specify particular expressions, gestures, or actions for individual subjects is an explicit stated goal.

• Scaling and reward design. Whether the RL stage scales gracefully to larger backbones or longer videos, and whether richer reward models can substitute for the hand-weighted composite used here, remain open questions.

Target Audience

Researchers and engineers working on diffusion-based generation, video synthesis, or RLHF-style post-training for generative models. The paper will also interest practitioners building subject-driven content tools who need to evaluate whether attention design or post-training is the better lever for identity fidelity. Readers without background in diffusion transformers or policy-gradient RL will find the architectural details dense, but the high-level comparison and ablation results are accessible to anyone familiar with generative video.

Authors’ abstract

Significant progress has been achieved in high-fidelity video synthesis, yet current paradigms often fall short in effectively integrating identity information from multiple subjects. This leads to semantic conflicts and suboptimal performance in preserving identities and interactions, limiting controllability and applicability. To tackle this issue, we introduce ID-Crafter, a framework for multi-subject video generation that achieves superior identity preservation and semantic coherence. ID-Crafter integrates three key components: (i) a hierarchical identity-preserving attention mechanism that progressively aggregates features at intra-subject, inter-subject, and cross-modal levels; (ii) a semantic understanding module powered by a pretrained Vision-Language Model (VLM) to provide fine-grained guidance and capture complex inter-subject relationships; and (iii) an online reinforcement learning phase to further refine the model for critical concepts. Furthermore, we construct a new dataset to facilitate robust training and evaluation. Extensive experiments demonstrate that ID-Crafter establishes new state-of-the-art performance on multi-subject video generation benchmarks, excelling in identity preservation, temporal consistency, and overall video quality. Project page: https://angericky.github.io/ID-Crafter