WorldFoundry: Towards Unified Infrastructure for World Intelligence

AI summary

Overview

Research area: Generative world modeling and computer vision, with a strong secondary emphasis on machine-learning systems infrastructure and benchmarking (video, 3D/4D, embodied-action, and interactive world models).

Technical level: Intermediate. Readers need some familiarity with generative models, latent state representations, and standard evaluation practice, but the paper is organized around a conceptual definition and an infrastructure design rather than new model architectures or training math.

Scope in one sentence: The paper proposes a functional definition of what counts as a generative world model and releases WorldFoundry, an open-source system that unifies discovery, execution, artifact management, and evidence-tracked evaluation for 270 models, 74 benchmarks, and 62 registered metric entries across video, 3D/4D, embodied, and interactive modalities.

What This Paper Is About

Visual generative models increasingly produce output that looks like a simulated environment, but good-looking video does not prove that a model carries persistent state, obeys physical dynamics, or responds causally to interventions. Because there is no shared operational basis and evaluation is perception-centric, the field struggles to separate "generative foundation models" from genuine "generative world models," and results are hard to reproduce or compare. The paper's goal is to supply both a definition that draws that boundary and the open-source infrastructure needed to execute and evaluate heterogeneous world models reproducibly.

Key Contributions

1. A formal definition of a generative world model. The authors define it as a learned, history- and control-conditioned dynamical system that infers and maintains a persistent environmental state, evolves that state under endogenous dynamics and external interventions, and assimilates observational feedback into coherent future trajectories.

2. A minimal functional abstraction plus a capability framework. The definition is decomposed into three operations — state inference, forward simulation, and interactive response — and expanded into an intelligence-oriented framework spanning perceptual grounding, persistent state modeling, controllable transformation, dynamics-aware prediction, and closed-loop interaction.

3. Unified inference and evaluation infrastructure. WorldFoundry standardizes execution and evaluation across video, 3D, 4D, embodied-action, and interactive world systems, using modality-specific adapters that preserve native inputs, execution semantics, output representations, and evaluation protocols rather than overriding them.

4. Validation at scale. The current catalog contains 270 model entries, 74 benchmarks, and 62 registered metric entries, with every model runner validated through end-to-end execution of a documented reference workflow, producing an auditable record of operational readiness under the tested configurations.

Main Findings

• Plausibility is not world modeling. The paper argues that visual plausibility reflects only the observation decoder and is neither necessary nor sufficient evidence that underlying state persists — a gap the authors say physics- and world-oriented benchmarks such as VBench2, VideoPhy2, WorldModelBench, and WorldArena repeatedly expose.

• Closure, not appearance, is the dividing line. A system qualifies as a world model only when an intervention leaves a trace in the state that constrains later predictions, and each incoming observation in turn revises that state. Foundational generative models synthesize or transform observations without closing this loop; world models couple generation to an evolving environmental state through closed-loop updates.

• A functional rather than architectural decomposition works across modalities. Static 3D methods emphasize persistent-state inference and geometry decoding, video models mainly learn implicit transitions and observation generation, 4D methods explicitly represent time-varying structure, and embodied models connect state estimation and action prediction to decision-making. Describing systems by which operations they implement gives a common vocabulary without forcing a single internal design.

• The bottleneck is infrastructural as well as algorithmic. Incompatible representations, runtimes, interfaces, and evaluation protocols make capabilities hard to compose and results hard to reproduce; the paper positions WorldFoundry as addressing that alongside efforts such as OpenWorldLib.

• Execution readiness is reported separately from evaluative claims. Declarative manifests bind discovery, runtime, and evaluation; durable artifact contracts with typed references decouple execution from evaluation; and evidence-aware scorecards separate declared support, operational readiness, and validated performance, so each claim traces to supporting artifacts.

• Catalog scale. 270 model entries, 74 benchmarks, and 62 registered metric entries, with integration readiness and execution evidence recorded separately for each entry. The introduction and contributions section describe this as "over 100 benchmark and metric implementations," consistent with 74 benchmarks plus 62 metric entries.

• No comparative performance numbers are reported in the provided content. The paper presents a comparison table (Table 1) against world-model infrastructures, modality-specific toolkits, and evaluation suites along coverage (video, 3D/4D, interactive world models, embodied), execution (model runner, hosted API, closed loop, training), and evaluation (protocols, evidence, world-model definition, interfaces), but the individual entries and any resulting scores are not included in the supplied text.

Methodology in Plain English

The authors start by asking what a world model must do, not how it is built. From prior reinforcement-learning definitions and recent scope-setting work, they settle on three jobs: estimate a hidden state of the environment from what has been observed, predict how that state changes over time and under outside actions, and produce distinct futures conditioned on those actions. They then argue that these jobs are only meaningful if something persists across them — a state that actions modify and that new observations correct — and they use that persistence requirement to separate ordinary generative models from world models.

With that definition in hand, they build an engineering layer around it. Rather than rewriting each model, they write per-modality adapters so each system keeps its native inputs, execution semantics, outputs, and evaluation protocol. Declarative manifests describe what a model needs and how it runs; a single runner executes runs either one-shot or as a persistent session; outputs are stored behind durable artifact contracts with typed references; benchmarks write their scores into those artifacts; and a scorecard reports what is backed by evidence versus merely declared. The stated design principle is that one run is reachable from every interactive and programmatic surface, so interactive visualization and scripted evaluation exercise the same execution path.

Validation is operational rather than comparative: every model runner is run end-to-end through a documented reference workflow, and that execution record is what backs the readiness claims.

Why This Matters

Impact on research. The paper tries to make "world model" an operational category rather than a marketing one. If adopted, it gives reviewers and readers a checklist — does state persist, do interventions leave traces, do observations revise state — and it makes claims auditable by tying each reported number to stored artifacts. That could reduce the current situation where generative demos and genuine dynamical simulators are compared as if they were the same kind of result.

Real-world applications:

• Robotics and embodied agents: embodied policies need action-conditioned, state-persistent predictions; a shared harness lets teams compare policies and simulators on the same footing before deployment.
• Simulation for autonomy and manufacturing: 3D/4D and video world models used to generate training and test scenarios require evidence that the simulated environment responds consistently to interventions, not just that frames look realistic.
• Content and media production: studios evaluating video generators for shot-consistent, controllable output can use the same persistent-state criteria to judge whether a tool supports iterative direction rather than one-shot clips.
• Model and benchmark governance: labs and standards bodies can use the declared-versus-validated scorecard split to distinguish "we integrated this model" from "we measured this model."

Industry relevance. The separation of execution readiness from performance claims, plus durable artifact contracts, maps onto how organizations actually need to audit third-party models. The catalog scale (270 models, 74 benchmarks, 62 metric entries) matters commercially because integrating and re-validating models is expensive, and shared infrastructure amortizes that cost.

Future Directions

• Populate the scorecard with comparative results. The provided content describes infrastructure and readiness evidence but reports no cross-model performance outcomes; the natural next step is using the harness to produce comparable dynamics, controllability, and interactive-utility measurements across systems.

• Extend the closure test into a measurable protocol. The paper defines persistence operationally — interventions must leave traces, observations must revise state — but the supplied text does not describe a concrete standardized probe for it. Turning that definition into an executable diagnostic is the open question at the center of the work.

• Broaden the catalog and keep it current. With 270 model entries and 74 benchmarks already listed and readiness tracked per entry, maintaining coverage as architectures change, and closing gaps in domains where Table 1 shows weaker support (such as closed-loop and training workflows), is ongoing work.

• Connect to adjacent integration efforts. The authors position WorldFoundry relative to OpenWorldLib and other toolkits and suites; aligning definitions and protocols across these projects would determine whether a genuinely shared operational basis emerges.

Target Audience

Researchers and engineers working on video generation, neural scene representation, 4D reconstruction, and embodied AI who need to compare heterogeneous systems fairly; benchmark and evaluation maintainers who want claims tied to artifacts; ML platform and infrastructure teams building internal model zoos and evaluation pipelines; and reviewers or program committees who need a principled way to judge whether a paper's system is a world model or a generative model with world-model framing.

Authors’ abstract

Visual generative models increasingly blur the line between synthesizing observations and simulating environments. Yet visual plausibility alone does not constitute successful world modeling: compelling outputs may lack persistent state, physical dynamics, or causally consistent responses to interactions. We define a generative world model as a predictive dynamical system that maintains a persistent, history-conditioned representation of an environment, updates that…