The Pulse

Researchers present infrastructure for comparing world models

A November 7 preprint introduces WorldFoundry, open-source infrastructure for running and evaluating generative systems across several modalities. Its catalog lists 270 model entries, 74 benchmarks and 62 registered metric entries, while tr

AI.info Team · Written with AI · fact-checked · 2026-11-10

An open-source project called WorldFoundry aims to make it easier to run and evaluate generative systems described as world models. The technical report, a preprint posted to arXiv on November 7, presents a common framework and infrastructure for comparing systems across video, 3D and 4D, embodied action, and interactive environments, according to its authors.

A definition based on persistent state

In the report, the authors define a generative world model as a predictive system that keeps a representation of its environment over time, conditioned on its history. It updates that representation in response to actions or interventions, then generates coherent trajectories reflecting outcomes under observed or counterfactual interventions.

The framework they derive covers perceptual grounding, persistent state modeling, controllable transformation, dynamics-aware prediction, and closed-loop interaction. The authors distinguish this goal from producing visually convincing outputs: a system may generate plausible images or video without maintaining state, modeling physical dynamics, or responding consistently to interactions.

The paper’s introduction describes state inference, forward simulation and interactive response as related functions. State inference estimates an environmental state from past observations; forward simulation predicts how it changes under dynamics and interventions; and interactive response produces action-conditioned futures. The authors say the abstraction describes functions, not a required architecture, and does not require every system to perform all three operations.

Catalogs, adapters and evidence

The catalog lists 270 model entries, 74 benchmarks and 62 registered metric entries. The report says integration readiness and execution evidence are recorded separately for each entry, distinguishing whether a system is ready to run from claims about its evaluated performance.

The paper presents the infrastructure as covering model and benchmark discovery, runtime and asset preparation, execution orchestration, artifact management and interactive visualization. The report says modality-specific adapters retain each system’s native inputs, execution semantics, output representations and evaluation protocols rather than forcing all systems into identical formats.

The authors say explicit manifests, durable artifact contracts and evidence-aware scorecards separate execution readiness from evaluative claims, making each claim traceable to its supporting artifacts. The paper describes a unified runner that can handle preflight requests either as one-off runs or as sessions, with outputs saved through typed artifact references. Benchmark scores are also stored in artifacts and reported in scorecards, according to the paper’s description.

Comparison across different systems

The paper frames infrastructure as part of the challenge: systems use different representations, runtimes, interfaces and evaluation protocols, which the authors say make capabilities difficult to compose and results difficult to reproduce. It groups static 3D methods, video models, 4D methods and embodied models by the operations they emphasize, rather than treating them as interchangeable.

It argues that a continuing loop—where interventions affect later predictions and new observations revise the state—is the relevant evidence for distinguishing world models from systems that generate or transform observations without that loop.

The authors say WorldFoundry is intended to support reproducible comparisons of persistent, controllable dynamics across heterogeneous systems. The paper presents it as infrastructure for evaluation; its catalog figures describe entries, benchmarks and registered metrics, with readiness and execution evidence tracked separately.