From Traces to Agentic Worlds: Agentic Language World Models for Interactive Environment Simulation

AI summary

Overview

Research area: Natural Language Processing / LLM agents — specifically language world models (LWMs) and trace-based environment reconstruction.

Technical level: Advanced. The paper assumes familiarity with reinforcement-learning-style environment formalisms, agent architectures, retrieval augmentation, and benchmark protocols for LLM agents.

Scope (1 sentence): The paper proposes and evaluates agentic language world modeling, in which a dedicated world model agent simulates an environment for a separate task agent using only a structured "worldbook" reconstructed from historical interaction traces, without rebuilding the original executable system.

What This Paper Is About

LLM agents are increasingly trained and tested in realistic environment replicas, but reproducing the original system — whether a terminal, a software workspace, or an enterprise internal tool — often requires code, data, or infrastructure that are simply unavailable. The paper's core idea is that historical interaction traces (actions, observations, failures, and their consequences) frequently remain accessible even when the system itself does not, and that these traces can be turned into a reusable behavioral specification instead of a rebuilt system. Trace2Env instantiates this: a world model agent serves as the environment for a task agent, consulting trace-derived knowledge plus persistent episode state to predict how the environment would respond to new actions.

Key Contributions

1. The agentic language world modeling paradigm. Rather than predicting the next observation from a single flattened prompt containing environment description, history, and action, the paper separates the task agent (which decides what to do) from a world model agent (which determines how the simulated environment responds). The world model agent has no direct access to the original environment and must actively seek trace-derived knowledge relevant to the current interaction.

2. Trace2Env, a learning-free reconstruction framework. It converts historical traces into an immutable environment worldbook with four linked components: schemas (action interface and representable state variables), grounded evidence (recorded transitions, selected demonstrations, and their original observations), induced abstractions (conditional effects, state constraints, observation contracts, descriptive conventions), and provenance (links from each artifact back to supporting traces and turns).

3. A persistent episode workspace separating knowledge by scope. The runtime workspace is a triple of the fixed worldbook K_E, a mutable episode-specific state ŝ_t, and an episodic memory M_t. The paper frames this as K_E describing how the environment behaves, ŝ_t describing what is currently true, and M_t recording what happened before. A missing state entry is treated as unknown rather than as evidence of absence.

4. A two-gate agentic simulation loop. The world model agent proposes a transition as a pair (candidate state effects Δ_t, proposed observation), and a shared harness validates and commits it. An applicability gate labels retrieved worldbook entries as supporting, uncertain, or format-only, explicitly separating relevance from applicability; a validation gate accepts or rejects the proposal by checking candidate effects on a state copy against schemas, rule support, and applicable state constraints.

Main Findings

• Next-observation prediction improves across environments and backbones. Trace2Env reaches the highest average judge score with both backbones: 75.94 with gpt-5.6-sol and 75.75 with deepseek-v4.1-flash, improving over Direct Prompting by 6.43 and 7.04 points respectively. It is best in 11 of 14 environment–backbone pairs and never underperforms Direct Prompting.

• Worldbook and runtime contribute complementary gains. On average scores, Direct Prompting reaches 69.51 (gpt) and 68.71 (deepseek); Trace RAG Prompting 73.76 and 73.06; Worldbook Prompting 74.35 and 74.86; Harness only 70.79 and 70.35. Structuring evidence outperforms supplying retrieved raw interactions, and Trace2Env exceeds the better of Worldbook Prompting and Harness only in 11 of 14 environment–backbone pairs.

• Per-environment scores show the largest gains where simulation is hardest. With gpt-5.6-sol, Trace2Env scores Terminal 65.42, SWE 70.59, Android 65.10, Web 59.55, Food 87.43, Shopping 85.36, and Benefits 98.11. With deepseek-v4.1-flash the corresponding values are 64.12, 70.17, 65.50, 57.85, 88.20, 86.07, and 98.33.

• High simulated success does not imply a faithful world model. In multi-turn interaction on ALFWorld, Direct Prompting reports Real 93%, WM 97%, W2R 3%, CR 0.032, while Trace2Env reports Real 93%, WM 88%, W2R 85%, CR 0.914. On SciWorld, Direct Prompting reports Real 85%, WM 92.5%, W2R 45%, CR 0.529, and Trace2Env reports Real 85%, WM 87.5%, W2R 60%, CR 0.706. Trace2Env sometimes looks worse inside the simulation but preserves far more real-environment validity.

• Evidence is the stronger standalone worldbook component; abstraction complements it. On Terminal, schema-only reaches 62.05, adding abstraction yields 60.69 (effect −1.12 ± 0.90), adding evidence yields 64.18 (effect +2.45 ± 0.89), and the full worldbook reaches 64.89. Abstraction given evidence adds +0.69 ± 0.63; evidence given abstraction adds +4.26 ± 1.02.

• More construction traces help, with rising one-time cost. The Terminal curve is less stable with 5–15 traces and improves steadily from 64.89 at 20 traces to 66.42 at 36 traces, still rising at the largest non-overlapping construction set available. The main experiments use 20 Terminal traces as a quality/cost trade-off.

• Gains extend beyond the construction data. The paper reports significant gains on Terminal, Web, and SWE records whose tasks are absent from the construction set, and trajectory-clustered confidence intervals for the gain over Direct Prompting exclude zero in every environment for both backbones.

• A single false transition can derail an entire interaction. In the qualitative ALFWorld case, Direct Prompting treats an unsupported put command as successful, commits a false placement, and produces an internally consistent but unreal rollout; Trace2Env predicts the correct no-op, preserves that tissuebox 2 remains in inventory, and lets the task agent recover with a valid move command.

Methodology in Plain English

Offline construction. Starting from recorded interaction traces, a construction model (gpt-5.6-sol for all worldbooks in the paper) aligns each action with its result and extracts observed facts, candidate state effects, outcomes, and uncertainty. Original observations are preserved; effects with ambiguous attribution are withheld. Schema induction establishes a common vocabulary for actions and states. The constructor proposes behavioral rules and reviews them against supporting and contrasting cases — a success–failure contrast can suggest a missing precondition, while a counterexample can narrow or render a rule tentative. Every derived artifact keeps provenance references. The finished worldbook is immutable, fixed for online simulation, and stored as a package reused across rollouts.

Online simulation. Each turn the world model agent starts from a compact view of the episode plus relevant worldbook knowledge, then selectively inspects more. Retrieved entries pass through the applicability gate (supporting / uncertain / format-only). The agent proposes a transition: an ordered list of declarative state-variable updates and a proposed observation. Candidate effects may follow reconstructed rules or be inferred through declared mutable state fields, so the simulator can operate even when no rule covers the case. The harness then runs the validation gate on a state copy; only accepted proposals are committed, with the observation resolved under applicable rendering constraints. Committed state and memory persist into the next turn, so continuity survives the fact that each action begins a fresh agent loop.

Evaluation design. Nine environments are split across two complementary settings. Single-step next-observation prediction uses the Terminal, SWE, Android, and Web splits of AgentWorldBench plus three EnvScaler environments (Food, Shopping, Benefits); scoring follows the official protocol with gpt-5.2 rating Format, Factuality, Consistency, Realism, and Quality on 1–5, rescaled to 0–100 and averaged. Multi-turn interaction uses ALFWorld and SciWorld, where a task agent solves each task once in the real environment and once in the simulated one; the action sequence produced under simulation is then replayed in the real environment, and the consistency ratio is CR = W2R / Real. Construction data comes from 20 own Terminal trajectories on Terminal-Bench 2.0, 50 WebArena trajectories, task-grouped five-fold cross-fitting for Android and SWE, 16 / 3 / 15 reserved EnvScaler trajectories, 40 ALFWorld trajectories, and 30 SciWorld gold-path trajectories. Deepseek results additionally test whether a worldbook built by one model can be operated by another, since no worldbook is reconstructed or adapted for the evaluation backbone.

Why This Matters

Impact on research. The paper reframes environment reconstruction: instead of reverse-engineering an executable system, it recovers a behavioral specification and simulates it. This connects two previously separate lines of work — LWMs that predict action consequences, and trajectory-distillation methods that mine skills and workflows — and positions the resulting artifact as a resource for a world model agent rather than for a task agent. It also argues that evaluating a world model by its internal success rate is misleading, and that real-environment replay (W2R, CR) is the criterion that matters for long-horizon simulation.

Real-world applications (drawn from the environments the paper studies):

• Agent training and evaluation for terminal and coding workflows, where the original sandbox, dependencies, or benchmark infrastructure may not be reproducible.
• Enterprise internal tools such as the employee-benefits portal and dietary-management platform studied via EnvScaler, where code, databases, and state snapshots are not available but interaction logs are.
• Web automation, using the Playwright MCP action interface and WebArena-style page snapshots, to rehearse agent behavior without repeatedly hitting live sites.
• Text-based decision environments such as ALFWorld and SciWorld, where a simulator that preserves state consequences supports grounded multi-turn agent development.

Industry relevance. The framework is learning-free — no model parameters are trained — and the same worldbook can be reused across backbones, which lowers adoption cost for teams that already log agent interactions. The reported cross-backbone result with deepseek-v4.1-flash on a gpt-5.6-sol-built worldbook is directly relevant to organizations that want a frozen knowledge artifact decoupled from whichever model they run.

Future Directions

• Strengthen active inspection and inference over the worldbook, since the paper notes fidelity remains bounded by what the traces reveal and that some backbones rely on reconstructed evidence less consistently than others.
• Expand reconstruction beyond observed behavior, moving past what the recorded traces directly demonstrate toward knowledge the traces only imply.
• Study agentic simulated environments as a substrate for task agent training and evaluation, rather than only as a simulation target.
• Reduce worldbook construction cost, since one-time construction cost grows roughly with trace data while fidelity was still rising at 36 Terminal traces — suggesting a trade-off worth optimizing.

Target Audience

Researchers and engineers working on LLM agents, world models, and agent environments; practitioners who need to test or train agents against systems they cannot reproduce; and readers interested in retrieval-augmented and agentic architectures for stateful, long-horizon simulation. It is most useful to those already comfortable with agent benchmark protocols, environment formalisms, and judge-based evaluation, and who want a concrete framework rather than a survey of the area.

Authors’ abstract

Realistic environment replicas are increasingly valuable for training and evaluating LLM agents, yet the original systems may be inaccessible or impractical to reproduce. We explore agentic language world modeling: rather than rebuilding an executable environment, a world model agent serves as the environment for a task agent and supports faithful and stateful simulation. We instantiate this paradigm with Trace2Env, a learning-free framework for settings where the original system is unavailable but historical interaction traces remain accessible. Trace2Env reconstructs these traces into a reusable environment worldbook containing environment schemas, grounded evidence, and induced behavioral knowledge. At runtime, the world model agent actively consults the worldbook together with persistent episodic state to infer each action's observation and lasting state effects. Across nine environments, Trace2Env improves both next-observation fidelity and long-horizon interaction consistency over conventional prompt-based LWMs. In multi-turn interaction, task agent actions generated against Trace2Env remain valid more often when replayed in the real environment, indicating that its simulated dynamics better preserve the consequences of earlier actions across successive turns. These results establish agentic language world modeling as an alternative direction for building realistic environment replicas without reconstructing the original executable system.