Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rollout

AI summary

Overview

• Research area: Autoregressive video diffusion models, specifically long-form / infinite-horizon video generation, positional encoding (3D Rotary Position Embedding), and inference-time KV cache control.
• Technical level: Advanced. The paper assumes familiarity with diffusion transformers, rectified flow, RoPE positional encodings, and KV caching, though the three proposed mechanisms are conceptually describable in plain terms.
• Scope (one sentence): The paper presents ∞-RoPE, a training-free, inference-time framework built on three components — Block-Relativistic RoPE, KV Flush, and RoPE Cut — that converts short-horizon autoregressive self-rollout video diffusion models into infinite-horizon, action-controllable, cinematic multi-cut video generators.
• Paper details: arXiv:2511.20649v3 [cs.CV], dated 19 Mar 2026 (published 2025-11-25). Authors: Hidir Yesiltepe, Tuna Han Salih Meral, Adil Kaan Akan, Kaan Oktay, Pinar Yanardag (Virginia Tech; fal). Project page: https://infinity-rope.github.io.

What This Paper Is About

Existing autoregressive video diffusion models face three bottlenecks: a finite temporal horizon imposed by the base model's 3D-RoPE (limited to a fixed maximum sequence length of 1024), slow responsiveness to new prompts during long rollouts, and an inability to produce discontinuous cinematic transitions such as scene cuts within a single generation stream. The paper's goal is to overcome all three without any retraining, new data, or long-video supervision, by reparameterizing how temporal positions and KV cache entries are handled purely at inference time.

Key Contributions

1. ∞-RoPE framework: A training-free methodology that converts existing short-horizon autoregressive self-rollout diffusion models into action-controllable infinite-horizon generators, applicable in a plug-and-play fashion on top of existing Self-Forcing variants.
2. Block-Relativistic RoPE: A relativistic positional encoding that reformulates temporal structure as a moving local reference frame. Each newly generated latent block is rotated relative to the base model's maximum frame horizon while earlier blocks are rotated backward to preserve relative temporal geometry, enabling generation far beyond the 1024-frame RoPE limit.
3. KV Flush: An inference-time operator that renews the KV cache by retaining only two tokens — the global sink latent frame and the last generated latent frame — providing prompt-responsive action control at constant memory and zero latency.
4. RoPE Cut: An inference-time operator that introduces controlled discontinuities in temporal RoPE coordinates, enabling cinematic multi-cut scene transitions within a single continuous rollout.

Main Findings

• Long-horizon quality on VBench: ∞-RoPE achieves an Overall score of 0.8377 on 5-second generations, 0.8298 on 60 seconds, 0.8236 on 120 seconds, and 0.8309 on 240 seconds. The paper reports that it "consistently ranks first or second in every metric" on the 120s and 240s benchmarks and achieves the strongest overall score.
• Consistency and motion in long rollouts: On 60-second videos, ∞-RoPE reaches Background Consistency 0.9490, Subject Consistency 0.9444, Motion Smoothness 0.9901, and Temporal Flickering 0.9852, described as superior long-horizon coherence and visual quality relative to prior autoregressive models.
• Dynamic degree advantage: On 120-second videos ∞-RoPE records Dynamic Degree 0.56 and on 240-second videos 0.64, values the paper highlights as sustained high motion richness, contrasting with baseline methods whose high temporal-quality metrics are attributed to stagnation reflected in low dynamic degree.
• Action-control user study: Compared against LongLive, SkyReels-V2, and Self-Forcing, ∞-RoPE obtains the best scores in Text Alignment (3.86), Subject Consistency (3.95), Motion Smoothness (3.74), and Video Quality (3.38), versus LongLive's 3.19 / 3.29 / 3.10 / 2.98.
• Long-video user study: With 50 participants recruited from Prolific rating 60-second generations on a 5-point Likert scale, ∞-RoPE achieves the highest average overall quality (3.911) and temporal consistency (3.708), average 3.810, outperforming the strongest baseline Rolling-Forcing (3.554 / 3.423, average 3.488), as well as CausVid (average 3.122), SkyReels-V2 (2.377), Self-Forcing (2.273), and NOVA (1.310).
• Mechanistic attention structure: Analysis of frame-to-frame attention maps from the 13th DiT layer shows Block-Relativistic RoPE produces a sharp diagonal band plus a persistent early-time sink column that remains stable beyond the original 3D-RoPE horizon; KV Flush suppresses attention to intermediate past frames and redirects mass toward the sink and the last few pre-flush frames; RoPE Cut splits the map into two nearly disjoint diagonal blocks.
• Cache size trade-off: Varying KV cache size at fixed f0 = 21 across 30s, 60s, and 120s sequences shows Imaging Quality and Dynamic Degree gradually decrease as cache grows, while Subject Consistency and Background Consistency remain stable.
• Temporal jump index sensitivity: For Δ = 6, 21, 45, 90 on 40-second videos with a scene cut every 10 seconds (20 videos generated), Subject Consistency falls from 90.74 (Δ=6) to 87.69 (Δ=90), Background Consistency from 88.98 to 82.27, and Temporal Smoothness from 0.98 to 0.95. In-horizon values (Δ = 6, 21) yield smoother transitions; out-of-horizon values (Δ = 45, 90) produce more dramatic scene changes with a visible transition edge from RoPE extrapolation.
• Throughput preserved: ∞-RoPE runs at 17.01 FPS, matching Self-Forcing, CausVid, and Rolling-Forcing, and far exceeding Wan2.1 (0.78), SkyReels-V2 (0.49), MAGI-1 (0.19), and NOVA (0.88).
• Semanticization of distant frames: When the cache exceeds f_limit, the earliest cached block B_3 = {1, 2, 3} is remapped to B_1̄ = {1, 1, 1}, collapsing temporally distant tokens into temporally invariant but semantically influential memory, compared by the authors to semanticization in cognitive neuroscience.

Methodology in Plain English

Rather than training anything new, the researchers take an existing causal video generator — a Self-Forcing model distilled from Wan2.1-T2V-1.3B that natively produces five-second videos at 16 FPS and 832×480 resolution — and change how it is driven at inference time.

The base model encodes each video's time, height, and width positions using 3D-RoPE, whose dimensions are configured with a maximum sequence length of 1024. The model, however, was only ever trained on 5-second clips, corresponding to f_limit = 21 latent frames, and generation proceeds in blocks of three latent frames. The key idea is to stop treating temporal positions as absolute. In Block-Relativistic RoPE, temporal coordinates are defined inside a moving local reference frame: new blocks are rotated relative to the base model's maximum frame horizon, and earlier cached frames are rotated backward so their relative temporal geometry is preserved within the fixed cache window. Once the cache grows past the model's horizon, the oldest frames have their coordinates collapsed to a shared minimum index, so they retain "what" happened rather than "when."

Two additional inference operators build on this geometry. KV Flush, triggered when a new prompt arrives, discards all cached tokens except two anchors — the global sink latent frame and the last generated latent frame — so the new action takes effect immediately without the latency of rebuilding the whole cache (as KV-Recache in LongLive requires) and without accumulated stale semantics. RoPE Cut performs a controlled discontinuous jump in temporal coordinates by re-mapping the current block from B_f = {f-2, f-1, f} to B_{f→f+Δ} = {f-2, f+Δ-1, f+Δ}, holding the first token as a local anchor while reassigning the next two to high-SNR positions; subsequent frames are generated as if the jumped segment were repositioned into past indices, letting the system reuse the base model's horizon without exceeding its temporal range.

Evaluation follows prior work using VBench (subject consistency, background consistency, motion smoothness, temporal flickering, dynamic degree, aesthetic quality, imaging quality) on prompts randomly sampled from MovieGenBench, generating over 100 videos across four durations: 5, 60, 120, and 240 seconds. Baselines include two bidirectional models (Wan2.1-1.3B, LTX-Video) and seven autoregressive models (Pyramid Flow, NOVA, SkyReels-V2, MAGI-1, CausVid, Self-Forcing, Rolling-Forcing). Implementation uses a KV cache size of 6, onset index f0 = 21, classifier-free guidance scale of 3.0, and timestep shift of 5.0. The supplementary material provides more than one hundred video results totaling over two hours of content.

Why This Matters

Impact on research: The paper argues that autoregressive video DiTs trained under the Self-Forcing paradigm on only 5-second clips already possess the capacity for highly dynamic infinite-horizon generation, and that the barriers are positional and cache-management artifacts rather than fundamental capability limits. Prior work extended horizons through longer self-rollouts, retraining on long-video data, or large-scale infrastructure; this work shows a training-free, plug-and-play alternative. It also offers a mechanistic account (via attention map structure) of why long rollouts stay stable, which is a useful diagnostic for the field.

Real-world applications:

• Interactive and streaming content creation where a user changes the described action mid-generation and expects immediate visual response.
• Narrative filmmaking and storyboarding requiring cinematic scene cuts, flashbacks, or cross-location transitions inside one continuous generation stream.
• Long-duration character-consistent video for advertising or social media, where identity drift and temporal degradation currently limit usable length.
• Rapid prototyping of multi-shot sequences without stitching separately generated clips.

Industry relevance: Because the method is training-free and inherits the throughput of the base generator (17.01 FPS, the same as Self-Forcing, CausVid, and Rolling-Forcing), it can be applied on top of already deployed distilled models without new data collection, fine-tuning budgets, or additional compute-heavy infrastructure. The constant-memory KV Flush and the latency advantage over KV-Recache are directly relevant to serving cost and responsiveness.

Future Directions

• Physical realism: The authors state that as a training-free method, ∞-RoPE inherits limitations of its base model, such as imperfect physics. Improving these would likely require model- or data-level changes rather than inference-time reparameterization.
• Out-of-horizon transition artifacts: Ablations show that temporal jump indices Δ = 45 and Δ = 90 fall outside the training horizon and produce a visible transition edge caused by RoPE extrapolation; reducing this artifact is an open problem.
• Cache size trade-offs: Larger KV caches were observed to gradually reduce Imaging Quality and Dynamic Degree even though Subject and Background Consistency stayed stable, leaving open the question of how to grow cache capacity without those quality costs.
• Scaling and generalization: The paper frames the work as a practical step toward future scalable, temporally robust video models and user-controllable long video generation; extending the approach across other base models and further refining relativistic temporal representations remain natural next steps.
• Comparison coverage: The paper notes that FLEX subsequently introduced frequency-aware RoPE modulation and antiphase noise sampling to mitigate spectral bias in long video generation, indicating an adjacent line of work that could be combined with or benchmarked against ∞-RoPE.

Target Audience

Researchers and engineers working on video diffusion models, autoregressive generation, and positional encoding methods; practitioners building interactive or streaming video generation systems who need long-horizon consistency and responsive action control without retraining; and anyone studying KV cache management strategies for autoregressive multimodal transformers. Readers need a working understanding of diffusion transformers and RoPE to follow the methodology sections in detail, but the conceptual contributions — a moving temporal reference frame, a two-token cache reset, and controlled positional discontinuities — are comprehensible to a broader technically literate audience.

Authors’ abstract

Current autoregressive video diffusion models are constrained by three core bottlenecks: (i) the finite temporal horizon imposed by the base model's 3D Rotary Positional Embedding (3D-RoPE), (ii) slow prompt responsiveness in maintaining fine-grained action control during long-form rollouts, and (iii) the inability to realize discontinuous cinematic transitions within a single generation stream. We introduce $\infty$-RoPE, a unified inference-time framework that addresses all three limitations through three interconnected components: Block-Relativistic RoPE, KV Flush, and RoPE Cut. Block-Relativistic RoPE reformulates temporal encoding as a moving local reference frame, where each newly generated latent block is rotated relative to the base model's maximum frame horizon while earlier blocks are rotated backward to preserve relative temporal geometry. This relativistic formulation eliminates fixed temporal positions, enabling continuous video generation far beyond the base positional limits. To obtain fine-grained action control without re-encoding, KV Flush renews the KV cache by retaining only two latent frames, the global sink and the last generated latent frame, thereby ensuring immediate prompt responsiveness. Finally, RoPE Cut introduces controlled discontinuities in temporal RoPE coordinates, enabling multi-cut scene transitions within a single continuous rollout. Together, these components establish $\infty$-RoPE as a training-free foundation for infinite-horizon, controllable, and cinematic video diffusion. Comprehensive experiments show that $\infty$-RoPE consistently surpasses previous autoregressive models in overall VBench scores.