Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation

AI summary

Overview

Research area: Generative computer vision, specifically long-horizon autoregressive video diffusion and distribution-matching distillation.

Technical level: Advanced. The paper assumes familiarity with diffusion models, reverse KL divergence, score functions, and autoregressive/kV-cache generation.

Scope: The paper proposes Rollout-Marginal Distillation (RMD), a two-stage training method that decouples visual-quality supervision from temporal context in autoregressive video generators, and evaluates it against Self Forcing and Causal Forcing on long-horizon video generation.

What This Paper Is About

Autoregressive video generators build long videos chunk by chunk, conditioning each new chunk on previously generated frames. Because every generated chunk contains small prediction errors, those errors become the context for future chunks and compound over long rollouts, causing visible degradation. Existing video-level distribution matching distillation (DMD) supervises the whole rollout jointly, so when the teacher scores a new chunk it also sees the flawed history and future, which forces the model to preserve artifacts for the sake of temporal consistency. The goal of this paper is to give the generator a clean, context-free quality signal while still keeping the video temporally coherent.

Key Contributions

1. Diagnosis of the coupling dilemma in video-level DMD. The paper formalizes how, in Equation 3, the gradient assigned to a chunk depends on the joint score of the entire sequence, so a correction toward natural appearance in the current chunk is penalized when it would break consistency with accumulated artifacts in the history.

2. The rollout-marginal objective. RMD pools continuation chunks across the student's causal rollouts into a chunk marginal distribution (Equation 4) and matches the initial frame and continuation chunks to separate teacher targets weighted by their frequency in the rollout (Equation 5), scoring each chunk independently of its history and position.

3. A practical two-stage training pipeline. Rollout-marginal training uses asymmetric denoising exits (initial chunk at a random intermediate step; continuation chunks only at the final lowest-noise step), a Wan 14B teacher adapted with LoRA on continuation chunks, and Checkpointed Self Forcing replay for backpropagation. A second stage then applies video-level DMD at smaller learning rates to restore cross-chunk temporal coherence.

4. Empirical demonstration of horizon extrapolation. RMD is trained on 5-second rollouts and evaluated on 60-second videos with a quantitative advantage over video-level DMD baselines, plus a qualitative 500-second example.

Main Findings

• Large margin on VBench-Long at 60 seconds. With chunk size 1, RMD reaches a total score of 81.26, quality 84.52 and semantic 68.24, versus Self Forcing at 70.94 / 77.89 / 43.17 and Causal Forcing at 69.88 / 78.34 / 36.03. With chunk size 3, RMD reaches 81.48 total, 85.03 quality and 67.24 semantic, versus Self Forcing at 77.44 / 81.99 / 59.22 and Causal Forcing at 74.88 / 80.53 / 52.27.

• Stable quality as the rollout lengthens. From 10 to 60 seconds, RMD's quality score moves only from 84.68 to 84.52 and it retains a semantic score of 68.24 at 60 seconds, while Self Forcing's total score drops from 79.12 to 70.94 over the same interval.

• The rollout-marginal objective is doing the heavy lifting for appearance. In ablations at chunk size 1, video-DMD-only scores 72.44 total and 78.68 quality and collapses into severe color artifacts, whereas the "marginal only" variant reaches 84.90 quality.

• Video-level refinement is needed for temporal coherence. Comparing "marginal only" to full RMD, subject consistency rises from 97.29 to 98.05, background consistency from 96.26 to 96.89, temporal flickering from 99.06 to 99.41, and motion smoothness from 98.79 to 98.96; qualitatively, the marginal-only variant shows jagged flickering in spatio-temporal slices, while video-DMD-only shows severe color artifacts.

• Asymmetric denoising exits matter. Applying random-step exits uniformly to all chunks gives 78.29 total and 82.48 quality, worse than full RMD's 81.26 and 84.52, and produces severe temporal jitter.

• No inference-time overhead or architectural change. Both baselines share the sliding-window protocol, so RMD introduces zero additional computational overhead at inference and uses no fixed frame anchors, no extra memory mechanism, and no generator modifications.

Methodology in Plain English

The researchers start from a pretrained bidirectional video diffusion model (Wan 14B) and distill it into a smaller causal generator (Wan 1.3B) initialized from Causal Forcing's causal ODE checkpoints. During training, the generator produces its own rollout of chunks, each sampled in 4 denoising steps, so it sees the imperfect history it will face at inference.

The key change is where the training signal comes from. Instead of asking a teacher to score the entire rollout at once, RMD scores each chunk on its own. Because a single continuation chunk looks statistically different from a video's first frame under Wan's causal VAE, the team adapts the Wan 14B teacher with LoRA on continuation chunks taken from videos generated by the original model, giving continuations their own target distribution. A frozen "real" score and an online "fake" score are compared on the same noisy chunk, and their normalized difference produces a gradient for that chunk. The initial chunk and the continuation chunks are handled separately and weighted by how often each appears in the rollout.

Two engineering choices make this work. First, supervision is asymmetric in denoising time: the first chunk is supervised at a random intermediate step, while continuation chunks are supervised only at the final low-noise step, which is where local detail is refined and where dropping temporal context does the least damage to the pretrained motion prior. Second, the causal rollout is generated without keeping its computation graph, and a differentiable replay recomputes each supervised prediction so the per-chunk gradients can be backpropagated; attention to the initial block is masked during replay in the latter half of the training window.

Because independent chunk scoring says nothing about how chunks fit together, a second stage refines the generator with ordinary video-level DMD, where the teacher and fake-score network judge the whole rollout jointly. This stage starts from the already-trained generator and uses smaller learning rates. Training uses 70,000 unique prompts, a fixed 81-frame horizon, and never performs longer rollouts than that during training. Evaluation is on VBench-Long with 944 unique prompts and five generations each, at 832×480 and 16 FPS.

Why This Matters

Impact on research. The paper reframes the long-horizon failure of autoregressive video models as a supervision-design problem rather than a memory or architecture problem. It shows that a pretrained bidirectional prior can be reused as a chunk-level target even when the surrounding generated context is degraded, and that quality and temporal coherence can be optimized in separate stages. This contrasts with approaches such as Context-Matched Distillation, which instead constructs a causal teacher conditioned on the student's realized history, and with methods that add persistent frame sinks or cache maintenance.

Real-world applications:

• Autonomous-driving simulation, where long continuous generated scenes are needed and accumulated artifacts would make downstream perception unreliable.
• Interactive and embodied world models, where low-latency streamable generation must stay coherent over long agent trajectories.
• Physics-grounded visual simulation, where structural collapse over time invalidates the simulated dynamics.
• Streaming content generation and long-form video synthesis, where the model must run well past the clip length it was trained on.

Industry relevance. RMD adds no inference cost, requires no architectural change, and works with a fully sliding context window, so it can be applied as a post-training recipe on top of an existing causal generator checkpoint. The training budget reported is modest by large-model standards (900, 750, and 800 steps at a global batch size of 8), which makes the approach practical to reproduce on top of existing Self Forcing or Causal Forcing pipelines.

Future Directions

• Quantitative evaluation beyond 60 seconds. The aggregate metrics cover roughly 60-second rollouts; the 500-second result is a single qualitative example. Whether RMD's stability holds quantitatively at that scale is not reported.

• Interaction with memory and cache mechanisms. RMD deliberately uses a fully sliding context window with no frame sinks or explicit memory. Whether it composes with techniques such as persistent frame sinks, Rolling Sink-style cache maintenance, or real long-video context is an open question.

• Extending the chunk-marginal idea to other modalities and architectures. The method is demonstrated on a causal Wan generator distilled from a Wan 14B teacher; its transfer to other backbones, chunk sizes, or autoregressive modalities is untested here.

• Reducing the cost of the second stage. The paper notes that independent chunk scoring creates inter-chunk motion jitter that must be repaired by a full video-level DMD refinement pass. Finding a way to obtain cross-chunk coherence without a separate stage, or characterizing how much refinement is needed, is a natural next step.

Target Audience

Researchers and engineers working on video diffusion, autoregressive or streaming generation, and diffusion distillation. It is most useful to readers already comfortable with DMD-style distribution matching and score-based training, and to practitioners building long-horizon world models, simulators, or streaming video generators who want a post-training recipe that requires no changes to inference or architecture. Readers looking for an introductory treatment of diffusion models will find the formulation dense.

Authors’ abstract

Autoregressive (AR) video diffusion enables low-latency, streamable video generation, but prediction errors often accumulate over long rollouts. Training the generator on its own rollouts exposes it to these imperfect histories. However, existing video-level distribution matching distillation (DMD) scores the whole rollout jointly. Because a chunk is evaluated together with its past and future, its correction can favor matching artifacts in the surrounding context merely to preserve temporal consistency. To provide a clearer visual-quality signal, we introduce Rollout-Marginal Distillation (RMD). RMD retains the generated history for AR prediction but scores each chunk independently against a chunk teacher, ensuring its quality correction is not compromised by an imperfect temporal context. To compensate for the lack of temporal context in independent chunk scoring, RMD subsequently applies video-level DMD to restore temporal coherence. Extensive experiments demonstrate that RMD maintains high visual quality far beyond its training horizon and outperforms video-level DMD baselines. Code and video results are available at https://cjeen.github.io/RMD