InternW0: A Foundational Physical World Model for Efficient Real-World Interactions

AI summary

Overview

• Research area: Robotics — physical world modeling and robot manipulation (arXiv:2609.27656v1 [cs.RO], published 23 Sep 2026).
• Technical level: Advanced.
• Scope: This paper introduces InternW0, the first model in the InternW series from Shanghai AI Laboratory, an asymmetric video–action world-action model trained on roughly 7,200 hours of robot and egocentric data that couples future visual prediction with continuous robot control through asynchronous, multi-frequency inference.

What This Paper Is About

Physical intelligence, in this paper's framing, requires that a robot's predictions about the world stay actionable while the world keeps changing — not just that those predictions are accurate. The authors argue that existing world-action models (WAMs) fall short on three counts: they do not cleanly support heterogeneous embodiments and richer physical modalities such as force and tactile sensing, and they suffer latency because expensive video prediction is updated in lockstep with fast action generation. InternW0 is the authors' proposed answer: a mixture-of-transformers architecture in which a heavy video expert predicts future visual dynamics on a slow schedule while a lightweight action expert generates short control chunks on a fast schedule, reusing and locally editing cached video context instead of regenerating the future for every action update.

Key Contributions

1. A framework and an initial instantiation. The paper lays out the InternW series-level design — omnimodal interfaces, asynchronous multi-frequency processing, and local modeling of environment and action under partial observation — and realizes it in InternW0, an asymmetric mixture-of-transformers (MoT) model with separate video and action experts jointly optimized by video–action flow matching.
2. An observation-conditioned chunk K/V editor for asynchronous duplex inference. Instead of recomputing the predictive video plan at every action update, InternW0 caches layerwise video keys and values and lets each action chunk adapt them using its newly observed visual state (drawing on the observation-guided video-context routing design of AHA-WAM), so the same predictive context yields chunk-specific views.
3. Contact-aware post-training with joint action–contact prediction. The action interface is extended with force and tactile histories, and the model predicts a combined target of action, force/torque, and tactile representation, linking observed interaction history to both commands and their anticipated sensory consequences.
4. A large heterogeneous data recipe and science-native evaluation. The corpus spans seven datasets totaling 7,233.5 hours and 811,969 episodes across 25 training domains, including EgoLab, a 275-hour egocentric real-laboratory dataset collected by the team. Evaluation extends beyond simulation to real scientific tasks: a 15-stage metal–organic framework synthesis workflow and 5-stage contact- and force-aware dexterous manipulation for general-purpose quantitative pipetting.

Main Findings

• Asymmetric dual-expert design. The video expert builds a shared predictive representation from visual observations and language without consuming robot-specific action tokens or domain identities, while the action expert reads that context together with the latest observation, proprioception, language conditioning, and embodiment-specific interfaces. The paper states this separation keeps physical prediction broadly shared while allowing control to specialize across embodiments.
• Latency is addressed by decoupling timescales. The video expert updates its prediction on a slower schedule; the action expert generates short chunks from the latest plan and current feedback. After an initial plan is generated, action inference can continue while the next video prediction is computed. The authors frame predictive accuracy, decision relevance, and timeliness as the three design criteria.
• The K/V editor starts as an identity mapping. Its final residual projection is zero-initialized and gated by a learned per-layer gate, so observation-dependent corrections are introduced progressively; the underlying video context itself remains unchanged.
• Action gradients flow into the video expert during training. In the independent-denoising (IDM) conditioning stream, the condition K/V features are explicitly not detached from the video network, so gradients from the action flow-matching objective propagate into the shared video expert. The condition stream is kept clean with probability 0.5, otherwise perturbed with an independently sampled shifted flow time; the first latent frame stays clean in both cases.
• Data composition is dominated by simulation and one real-robot source. By the paper's Table 1, InternData-A1 contributes 3,494.3 hours across 568,194 episodes (6 domains) and AgibotWorld 2,620.0 hours across 158,380 episodes (2 domains). RoboCOIN contributes 438.1 hours (13 domains, 59,952 episodes), Galaxea 314.9 hours (1 domain, 15,362 episodes), EgoLab 275.4 hours (1 domain, 3,192 episodes), MolmoAct 70.3 hours (1 domain, 3,424 episodes), and RoboDojo 20.5 hours (1 domain, 3,465 episodes).
• Domain mixture uses square-root weighting. Each domain is sampled with probability proportional to the square root of its number of effective phase-0 anchors, following the sublinear reweighting principle the paper attributes to RDT-1B (square root of dataset size) and π₀ (m^0.43). Each GPU batch contains samples from a single domain, which the paper says is required by the per-domain action interfaces.
• Unified 37-dimensional interface. Each arm occupies 17 dimensions (7 joint values, 3 end-effector translation, 6 orientation in rotation-6D, 1 gripper), with the remaining 3 dimensions reserved for the mobile base. Head and waist degrees of freedom are outside the supervision space.
• Temporal convention and sample shape. All datasets are subsampled to a 15 fps model-time convention: stride 4 for 60/50 fps streams, stride 2 for 30/25 fps streams, and original stride for 15 fps streams. Each robot training sample holds 13 multi-view frames stitched into a 384×320 canvas and 64 action steps, with consecutive frames separated by 8 control steps so the 13 frames span 96 control steps; the action sequence is split into four 16-step chunks.
• LIBERO baseline numbers are reported in the paper. In the LIBERO Full setting (50 rollouts per task), the reported averages are: π₀ 94.4, InternVLA-M1 95.9, π₀.₅ 96.9, GR00T-N1.7 97.0, OpenVLA-OFT 97.1, and Fast-WAM 97.6 (Spatial 98.2 / Object 100.0 / Goal 97.0 / Long 95.2). Fast-WAM is the highest average among the baselines listed. The Motus row begins at 96.8 on Spatial and is cut off in the provided content.
• InternW0's own benchmark scores are not reported in the provided content. The text of the full paper supplied here is truncated partway through Table 2, so InternW0's LIBERO suite scores and macro-average, the RoboTwin 2.0 results for the Full and Clean2Random regimes, and all real-world experiment outcomes are not available to report.

Methodology in Plain English

The system is built as two cooperating networks sharing a joint layer structure. A large video expert takes encoded video frames plus language and predicts future visual latents. A small action expert takes the same language, the robot's proprioceptive state, embodiment-specific inputs, and — importantly — context read from the video expert to generate a chunk of robot actions. Visual encoding uses a frozen Wan VAE for video frames, a frozen DINOv3 encoder for chunk-level observations, and precomputed text embeddings; only the projections into the experts are trainable.

The key trick is that the video expert does not need to re-run for every action chunk. Its layerwise keys and values are cached. For each new action chunk, the most recent visual observation is encoded and summarized into a few routing queries, which retrieve task-relevant information from the cached video representation; that retrieved information is then routed back onto the original video token positions to produce small residual corrections to the keys and values. Because those corrections are zero-initialized, the model begins by ignoring the observation and learns over training how much to adapt. Each chunk therefore gets its own edited view of the same prediction, and control can keep running while the next prediction is being computed.

Training supervises both experts with continuous flow matching — noise is interpolated with clean targets, and the network learns the velocity pointing from noise to data — with shifted flow-time sampling, per-chunk independent flow times, and a first-frame-causal attention mask where the observed first frame anchors the sequence and sits outside the video loss. Egocentric videos with no action labels contribute video supervision only; robot trajectories supervise both experts. A second, independently noised video stream supplies the conditioning features used by the action expert, and gradients from the action objective back-propagate into the video expert.

For contact-rich tasks, a post-training stage adds force and tactile histories as action-side observations and extends the prediction target to include force/torque and tactile representations alongside actions, with future contact channels noised and denoised jointly with actions while only past contact history is given as conditioning.

The engineering pipeline is organized as three planes: a data plane that curates heterogeneous episodes into unified streams (shuffling compact metadata rather than payloads, materializing payloads on demand, caching decoded video node-locally, and generating sample-level randomness deterministically so runs are reproducible), an execution plane built on Ray for cluster management and co-locating data materialization with GPU training, and an optimization plane using composable FSDP2 sharding at the level of joint MoT layers and expert stages, FP32 master parameters with BF16 computation and FP32 gradient reduction, full-graph compilation per joint layer, selective activation checkpointing, and transactional distributed checkpointing that supports exact mid-epoch resumption.

Why This Matters

• Impact on research. The paper pushes world-action modeling toward asynchrony as a first-class design concern. Rather than treating video prediction and action generation as a single synchronized loop, it makes the rate mismatch between them explicit and solves it with a caching-plus-editing interface. It also argues that predictive representations should be shaped by their usefulness for control, evidenced by gradient flow from the action objective back into the video expert. Its scale of pretraining data (7,233.5 hours across 25 domains) and its explicit handling of heterogeneous embodiments via soft prompts and per-domain interfaces point toward how robot foundation models may be trained and deployed across mixed fleets.
• Real-world applications (as described in the paper):
  • Automated laboratory chemistry, framed around a 15-stage metal–organic framework synthesis workflow connected to the InkStone scientific discovery platform and Intern-S2-Preview for reasoning and tool use.
  • General-purpose quantitative pipetting, implemented as 5-stage contact- and force-aware dexterous manipulation.
  • Contact-rich manipulation more broadly, where force and tactile feedback carry information that may not be observable in images — the paper cites pipetting, sample weighing, titration, filtration, mortar-and-pestle grinding, stirring, and rinsing glassware among the EgoLab activity categories.
  • Cross-embodiment deployment, where soft prompts and domain-specific encoders/decoders let one shared backbone serve robots with different control spaces and sensor configurations.
• Industry relevance. The latency argument matters directly for any production robot: recomputing a full video prediction at every control tick is expensive, and executing against a stale plan is unsafe as the world drifts. An architecture that reuses one prediction across many action chunks, and that revises local context from fresh observations, is a practical route to running large predictive models on real hardware. The paper's emphasis on using experimental execution records to support subsequent model improvement also sketches a feedback loop for continuous improvement in industrial and scientific settings.

Future Directions

• The paper explicitly frames InternW0 as an initial instantiation and refers to the future work of the InternW world model series, leaving open how later members extend omnimodal interfaces, asynchronous processing, and local environment modeling.
• The out-of-domain question the authors set up through RoboTwin 2.0's Clean2Random regime — whether physical representations survive when the evaluation distribution shifts beyond the demonstrations used for adaptation — is posed as a core evaluation goal; the corresponding results are cut off in the provided content, so how well InternW0 actually generalizes beyond its adaptation data remains unanswered here.
• Contact-aware post-training currently extends only the action-side interface while preserving the shared video backbone, and force/tactile channels are optional. How far contact modalities should be integrated into the predictive backbone itself is left open.
• The duplex loop depends on timestamp alignment and causal consistency, so that each update accounts for already-executed actions while still being able to revise commands not yet issued. The paper frames this as a correctness requirement of the design; the mechanisms for enforcing it under real asynchronous hardware loads are not detailed in the provided content.

Target Audience

This paper is aimed at robotics and embodied-AI researchers working on world models, vision-language-action models, and robot foundation models, particularly those concerned with inference latency, asynchronous control loops, and cross-embodiment training. It will also be relevant to engineers building manipulation systems that depend on force and tactile feedback, and to researchers applying AI to laboratory automation and scientific experimentation. Readers without a background in flow matching, mixture-of-transformers architectures, or distributed training infrastructure will find the middle sections demanding.

Authors’ abstract

Physical intelligence requires more than predicting how the world may evolve: predictions must remain actionable as the world continues to change. We introduce InternW0, the first instantiation of the InternW physical world model series from Shanghai AI Laboratory, built around omnimodal interfaces, asynchronous multi-frequency processing, and local physical modeling under partial observations and external influences. InternW0 jointly learns future visual dynamics and continuous robot control through an asymmetric video--action architecture with flow matching. A high-capacity video expert provides longer-horizon predictive context, while a lightweight action expert operates at a faster timescale. Instead of regenerating the future for every action update, InternW0 reuses layerwise K/V and adapts it to newly observed states through observation-conditioned context routing. Domain-specific interfaces and soft prompts support heterogeneous embodiments, while contact-aware post-training incorporates force and tactile signals for contact-rich manipulation. We train InternW0 on approximately 7,200 hours of heterogeneous robot and egocentric data, including EgoLab, a 275-hour real-laboratory egocentric dataset. Evaluation spans simulation benchmarks and real-world scientific tasks, including a 15-stage metal--organic framework synthesis workflow and 5-stage contact- and force-aware dexterous manipulation for general-purpose quantitative pipetting. These results advance scalable, asynchronous, and science-native physical world models for universal and efficient real-world interactions.