Multics: Generative Multi-Player World Model with Hierarchical Communication

AI summary

Overview

Research area: Computer vision / generative video world models — specifically multi-player interactive video generation with structured inter-player communication.

Technical level: Advanced. The paper assumes familiarity with diffusion transformers, self-attention masking, cross-attention conditioning, and multi-agent communication topologies.

Scope: The paper proposes Multics, a multi-player video generation framework that replaces fully-connected or centralized inter-player communication with a three-level player–group–global hierarchy, and evaluates it on 4- and 6-player training plus 6-, 8-, and 10-player inference.

What This Paper Is About

Multi-player world models try to simulate a shared environment from the viewpoints of several players at once, so that each generated video agrees with the others where views overlap while still obeying its own player's actions, camera motion, and prompt. Existing approaches break down in two ways: fully-connected communication (Solaris) lets every player attend to every other, which scales quadratically with player count, while centralized communication (γ-World) routes everything through one shared global representation, which mixes unrelated concurrent events and degrades per-player control. Multics aims to get both robustness and scalability by inserting an intermediate group level between players and the shared global tokens.

Key Contributions

1. Hierarchical communication topology. Multics structures information exchange across player, group, and global levels, inserting group tokens into self-attention as intermediate group-level representations that aggregate locally relevant player information and connect to global tokens for cross-group exchange. Players in the same group can attend directly to one another; ungrouped players communicate through the global tokens.

2. Shared group-token parameters. A single set of learnable group-token parameters (E^L ∈ R^{k_L × d}) is replicated across all group slots and broadcast over latent frames. Because the slots share parameters and are exchangeable, the same communication module accommodates varying player counts without new group-specific parameters or retraining. An ablation of shared group-token parameters is reported in the paper's Appendix C.2.

3. Budget-constrained dynamic grouping. Player relevance is estimated from recent camera trajectories using two geometric cues — scene relevance (camera proximity and viewing-direction alignment) and mutual-view relevance (whether players fall inside each other's fields of view) — combined as a max and aggregated over a causal history. Under a communication budget C_max matched to the centralized baseline, players are grouped to maximize coverage of high-relevance relations, with group slots M = ceil(log₂ N).

4. A unified evaluation across player counts and domains. The paper reimplements baselines on the same base model and dataset, reports 10 metrics across consistency, action accuracy, camera accuracy, and visual quality in 4- and 6-player settings, tests generalization from 4-player training to 6-, 8-, and 10-player inference, and evaluates cross-domain on EgoTeam.

Main Findings

• State-of-the-art in 4- and 6-player settings. Multics outperforms Solaris and γ-World on most metrics in both settings. At 4 players it reaches GeoKPM 54.17% (vs 48.51% Solaris, 49.42% γ-World), IDM 94.17 (vs 93.87, 93.98), RotErr 2.434 (vs 2.572, 2.728), FID 26.39 (vs 28.41, 29.36), and FVD 162.3 (vs 171.9, 200.5). At 6 players it reaches GeoKPM 56.84% (vs 54.65%, 52.80%), FID 16.34 (vs 18.83, 20.16), and FVD 101.2 (vs 115.1, 128.4).

• Dense communication does not guarantee better generation. Solaris, despite unrestricted cross-player exchange, underperforms Multics on most metrics while incurring substantial computational overhead.

• Centralization degrades control under concurrency. γ-World struggles with consistency and camera accuracy because heterogeneous player information is aggregated through one shared global pathway.

• Pixel-aligned metrics tell a different story than geometric ones. PSNR and SSIM compare pixel values or local statistics at corresponding spatial locations and are sensitive to small spatial deviations; the paper argues the consistent gains in GeoKPM and LPIPS indicate stronger geometric and perceptual cross-player consistency.

• Generalization without retraining. Models trained with 4 players are applied to 6-, 8-, and 10-player inference with no additional training or parameters. At 4→10 players Multics reaches GeoKPM 51.54% (vs 47.82% Solaris, 49.12% γ-World), IDM 5.941 (vs 7.320, 7.138), FID 19.23, and FVD 121.9; baselines degrade more sharply.

• Computational efficiency. Fully-connected communication scales quadratically with player count, while Multics uses 9.6 TFLOPs versus 69.1 TFLOPs for the fully-connected alternative with 10 players, remaining comparable to centralized and group-based methods.

• Dynamic grouping beats alternatives. Ablations on 4-player OpenCS2 show GeoKPM of 54.17% for dynamic grouping versus 50.71% for fixed grouping, 48.26% for group-based communication, and 46.76% for random grouping. Random grouping also drops camera accuracy sharply (RotErr 14.61, TransErr 17.90, CamMC 29.83).

• Group-based communication alone is insufficient. Restricting local communication without an intermediate group-level representation causes inconsistencies when revisiting previously observed scenes and across player views.

• Budget sensitivity is asymmetric. Trained at a default budget of 1.0×, inference budgets from 0.6× to 1.4× were tested without retraining: dropping below 1.0× decreases GeoKPM, while increasing beyond 1.0× leaves GeoKPM relatively stable.

• Cross-domain applicability. On EgoTeam, Multics reaches GeoKPM 58.62% (vs 57.65% Solaris, 52.31% γ-World), IDM 97.07, RotErr 0.2894, TransErr 1.232, CamMC 1.381, FID 34.18, and FVD 195.1.

• Qualitative behavior. Multics shows stronger cross-player consistency on overlapping views, better adherence to player-specific controls such as aim-down-sights and no-firing, and greater interference robustness, preserving a player's scene on revisit despite other players' movements.

Methodology in Plain English

Multics is built on a video diffusion transformer (Wan2.2 TI2V 5B) that jointly denoises N players' videos. Each player i is conditioned on an initial image I_i, a text prompt P_i, an action sequence A_i split into 23-dimensional discrete controls and 2-dimensional relative yaw/pitch changes, and a camera trajectory C_i ∈ R^{f×3×4}. All players' camera rotations and translations are expressed in a shared coordinate system anchored at the first player's initial frame, and injected into that player's query and key features through camera adapters; group and global tokens receive zero camera features as placeholders. Actions use a causal window of r·w_c steps, discrete controls injected via cross-attention and mouse controls fused through a lightweight temporal module. Text is injected via cross-attention.

The heart of the design is the attention mask. Fully-connected communication concatenates all players along the sequence dimension and lets every player attend to every other. Centralized communication adds shared global hub tokens as a common interface. Group-based communication restricts direct player-to-player attention to players in the same group. Multics concatenates players, group tokens h^L, and global tokens h^G, and enforces the path x_i ↔ h^L_{g(i)} ↔ h^G, letting same-group players attend to each other directly and ungrouped players talk through the global tokens. Every group slot instantiates the same learned group-token parameters, so slots are interchangeable and the module handles different player counts.

Which players share a group is decided dynamically. At each grouping update, a symmetric relevance matrix is computed from camera positions, viewing directions, and mutual visibility using only past and current camera states to preserve causality, then temporally aggregated with recency weighting and smoothing. The number of group slots is M = ceil(log₂ N), and the maximum group capacity K is derived from the communication budget left over after subtracting the base hierarchical cost from the centralized cost (full formulas in the paper's Appendix B). Remaining slots and budget accommodate unassigned players, which otherwise communicate only through global tokens. Relevance decides topology only; the actual features exchanged are learned by self-attention.

Training uses diffusion forcing with a causal block of 3 latent frames: 10K steps with a 7-latent-frame local attention window, then 25K steps with 21 frames, learning rate 3×10⁻⁵, AdamW, 16 GPUs at per-GPU batch size 1 (effective batch size 16), bfloat16 with gradient checkpointing and gradient clipping norm 1.0. The VAE, text encoder, feed-forward layers, and remaining pretrained DiT parameters are frozen; self-attention, prompt cross-attention, action control modules, camera Q/K adapters, and the learnable group and global token embeddings are optimized, with 0.1 dropout on camera and text conditions. Multics uses 5 group tokens per group and 6 global tokens. Inference produces 165-frame videos at 352×640 with 50 denoising steps and classifier-free guidance scale 5.0. Simplex rotary encoding for players is applied to Multics and all baselines for permutation equivariance.

Why This Matters

Impact on research. The paper reframes multi-player world modeling as a communication-topology problem inside learned visual representations, an axis it describes as orthogonal to shared-world-state methods that scale via state–rendering decomposition. It provides a concrete alternative to the fully-connected versus centralized dichotomy, and shows that a training-time player count can be exceeded at inference without retraining or new parameters.

Real-world applications (drawn from the applications the paper cites for world models):

• Game entertainment and playable multi-agent game simulation, using the OpenCS2 Counter-Strike 2 setting as the training domain.
• Interactive simulation and interactive environments.
• Robotics, where ego-centric multi-agent observation is the norm, tested here on EgoTeam's multi-robot egocentric videos in reconstructed real-world indoor environments.
• Autonomous driving and physical simulation, listed as domains where video world models are being explored.

Industry relevance. The work comes from The University of Hong Kong and ARC Lab, Tencent IEG, and builds on a production-scale video backbone (Wan2.2 TI2V 5B). Its central selling point for deployment-oriented settings is cost: 9.6 versus 69.1 TFLOPs at 10 players, plus generalization to unseen player counts without retraining.

Future Directions

• Scaling beyond 10 players. The paper demonstrates 4→6, 4→8, and 4→10 inference but does not report results at higher player counts or the point at which the M = ceil(log₂ N) slot schedule becomes limiting.

• Richer relevance estimation. Grouping currently depends on camera trajectory geometry (proximity, viewing alignment, mutual visibility) only; whether visual content, actions, or learned relevance signals would improve grouping is not reported.

• Latency reporting. The paper states it compares DiT latency, self-attention latency, and self-attention FLOPs, but the text reports only the 10-player TFLOPs comparison (9.6 vs 69.1); the latency figures and their dependence on player count are in the figures rather than reported as numbers.

• Joint training across player counts. Because separate models are trained for N ∈ {4, 6}, and generalization comes from inference-time group replication, a single model trained across player counts, and its interaction with dynamic grouping updates, remains an open question.

Target Audience

Researchers and engineers working on generative world models, video diffusion transformers, and multi-agent simulation who need to understand how communication topology affects consistency, controllability, and cost in multi-player generation. It is most useful to readers already comfortable with attention masking and diffusion transformer training; readers new to the area will find the topology comparison in the overview figure the most accessible entry point, while the budget derivation in Appendix B is aimed at practitioners implementing the grouping scheme.

Authors’ abstract

Multi-player interactive world models aim to simulate shared environments from multiple players' perspectives, but achieving robust generation while scaling to more players remains challenging. Existing approaches that rely on fully-connected communication (e.g., Solaris) incur quadratic attention costs as the number of players grows, while centralized approaches (e.g., $γ$-World) suffer from interference between unrelated events in a shared global representation. To address these challenges, we present Multics, a multi-player video generation framework with hierarchical communication that structures information exchange across player, group, and global levels. This hierarchy introduces group tokens into self-attention as intermediate group-level representations that aggregate locally relevant player information. These tokens connect to global tokens for cross-group communication, while the hierarchical topology limits direct mixing between unrelated local events. Group-token parameters are shared across group slots, allowing the same communication module to accommodate varying numbers of players without introducing new group-specific parameters. We further propose budget-constrained dynamic grouping based on recent camera trajectories, which adaptively groups relevant players while limiting unnecessary communication. Experiments demonstrate that Multics achieves state-of-the-art performance in 4- and 6-player settings, with improved cross-player consistency, player-specific action and camera controllability, and interference robustness. Moreover, Multics generalizes from 4-player training to 6-, 8-, and 10-player inference without retraining, while reducing self-attention computation compared with fully-connected communication.