Twin: Playing an Unknown Game with a Test-Time Digital Twin

AI summary

Overview

Research area: Artificial intelligence — agentic systems, world-model learning, program induction, and interactive reasoning benchmarks (specifically ARC-AGI-3).

Technical level: Intermediate to Advanced. The paper is written for readers comfortable with reinforcement-learning framing (episodic MDPs, exploration vs. exploitation), program synthesis, and benchmark scoring, though the central idea is described in accessible terms.

Scope: The paper introduces and evaluates Twin, a system in which a coding agent writes an executable, validated world model of an unknown grid-world game at test time and plans inside it, scoring 93.3 out of 100 on the ARC-AGI-3 action-efficiency benchmark across 25 public games.

What This Paper Is About

The paper tackles the problem of an agent dropped into an unfamiliar game with no instructions: it does not know the controls, the rules of the world, or what counts as winning. The goal is to build an agent that infers both the dynamics and the hidden goal of the game from interaction alone, and then plays it as efficiently as a human playing for the first time.

Key Contributions

1. Test-time world-model inference. The agent writes the simulator it plans in — a Python program implementing step(grid, action) -> grid and goal_reached(grid) -> bool — turning the unknown environment into an executable hypothesis that can be inspected, falsified, repaired, and planned through, rather than a black box to react to.

2. Harness-enforced replay validation. Before any scored action is submitted, the harness requires the twin to reproduce every transition in the interaction history. This check is enforced by the harness rather than left to the agent, and a mismatch becomes a counterexample used to repair the model.

3. Pre-reward goal hypothesis. Twin proposes rival goal predicates before any reward arrives and discriminates between them by finding the most efficient plan, rather than waiting for a first reward to fit a goal. A candidate must evaluate false on every logged frame, and an exactly reached non-goal is permanently excluded.

4. Evaluation on the full ARC-AGI-3 public set. Twin clears 179 of 183 levels (97.8%) and 23 of 25 games, using fewer actions than humans on 158 of the 179 levels it clears (88.3%).

Main Findings

• Score of 93.3 out of 100. Twin reaches a mean score of 93.3 on the 25-game, 183-level public set, clearing 23 of 25 games and reaching the 100.0 ceiling on 18 games. For comparison, the base model played directly scores 7.8%, an off-the-shelf coding harness (Codex) scores 61.1%, OPINE-World scores 78.4, Prime Agent scores 78.3 with the same base model (GPT-5.6 Sol), and EWM scores 63.8. The human reference row is 100.0, 25 games, 183 levels.

• The harness, not model knowledge, accounts for the gain. Played directly, the base model clears one game and scores 7.8; the base model plus an off-the-shelf harness scores 61.1 with 13 games won and 148 levels cleared. The paper states the Twin harness accounts for the remaining 85.5 percentage points.

• Validated planning reduces scored actions. On the 13 games that Twin, EWM, and OPINE-World all fully clear, Twin uses 3,357 scored actions versus 5,367 for OPINE-World, 5,381 for EWM, and 7,485 for the human reference. Twin uses the fewest actions on 11 of these 13 games, matches or beats the human action count on 21 of its 23 cleared games, and uses 0.61× as many actions as humans on average.

• Most scored actions come from simulated plans. Of the scored actions, 92.9% execute a route already tried in the twin, while 7.1% are deliberate probes chosen to improve the model. Outcomes disagree with the twin on 20.1% of actions, and each mismatch becomes a counterexample for repair.

• Repairs generalize. 31 previously mispredicted situations recur later in play, and the repaired twin predicts all 31 correctly.

• Dynamics generalize better than novelty suggests. The twin predicts 8,210 of 10,392 first-seen state–action pairs exactly (79.0%), compared with 602 of 635 recurring pairs (94.8%) — a 15.8 percentage-point cost for novelty. On a separate appendix experiment, the final twins predict the complete next frame exactly for 70.1% of previously unobserved pairs, with per-game exact-match rates ranging from 9.9% (wa30) to 98.8%, and tu93, sp80, bp35, and ar25 exceeding 91%.

• Goal inference is the harder half. The first committed goal hypothesis is correct before any reward on 156 of the 179 completed levels (87.2%); search covers the rest. Goal errors are less frequent but costlier than dynamics errors.

• Where the cost is concentrated. The 23 non-optimal levels add 1,291 actions over the human baselines, and five levels account for 71% of that. Twin uses 1.5× human actions on tn36 and 1.6× on ka59.

• Hard games separate the systems. Easy games show no separation — all four systems score 100.0 on ar25 and cn04. Twin is the only system to clear bp35, lf52, and sk48, which contain nine, ten, and eight levels. On dc22, Twin clears all six levels in 1,219 actions, while EWM uses 1,842 actions and clears four.

• Failure cases invert the comparison. OPINE-World clears the two games Twin leaves unfinished. On sc25, Twin scores 32.7 against OPINE-World's 84.0, sinking 647 of its 701 actions into a hidden countdown on level 4; the ablation scores 44.8 there, clearing five levels by trial and error. On sp80, Twin scores 82.1 against 100.0 while its twin stays 92.3% accurate — a goal wall, not a dynamics wall. Conversely, Twin reaches the 100.0 ceiling on bp35 and lf52, where OPINE-World scores 2.6 and 4.2.

• Compute. Across the 25 runs, Twin used 2.60 billion processed tokens and 91.4 hours of wall-clock inference, averaging roughly 224,000 tokens per scored action. Per-game usage ranged from 5.1 million to 625 million tokens, and the ka59 run consumed 24% of all tokens for less than 10% of all scored actions.

Methodology in Plain English

The system treats each game as an unknown deterministic world. The agent starts with an "identity twin" — a stub program that predicts nothing changes — and an empty log. It plays through a loop with four parts:

• Validate. Every real move is appended to a transition log as a (state, action, next-state) triple. The twin must replay every entry. If it cannot, no scored action is allowed until it can.
• Explore. The agent is stuck in one of two ways: a dynamics wall (the twin can't replay the past) or a goal wall (it replays everything but finds no goal). At the dynamics wall, failing transitions are compiled into a ranked bug report and the coding agent repairs the code. At the goal wall, the harness ranks reachable states by signs of progress (a color appearing or disappearing, a compact region changing, a global scene change, reaching a new frontier) and proposes goal candidates.
• Plan. Breadth-first search runs inside the validated twin, using the predicted transition function and the hypothesized goal test. Search budgets are fixed across games: depth 8 with 20,000 nodes, widened to 14 and 30,000 for goal discovery.
• Execute checked. The planned route is submitted one move at a time against the real simulator. The executor halts at the first mismatch, which is logged as a counterexample; a match is verified progress.

Goal discovery works by hypothesis testing before any reward exists. When planning finds no route, the coding agent writes a tentative goal_reached predicate for the top-ranked candidate state. Two free filters apply before any scored action: the predicate must evaluate false on every logged frame, and a candidate previously reached without completing the level is permanently excluded. If a level boundary occurs, the hypothesis is confirmed; if the candidate state is reached exactly with no boundary, it is rejected. If reality mismatches the twin instead, that tests the dynamics, not the goal.

The system is driven by OpenAI Codex running the base model (GPT-5.6 Sol), connected to the game only through files with no game-specific tools. Scoring is the benchmark's action-efficiency metric between 0 and 100, which rewards clearing games in as few actions as a first-time human; reasoning, code, and simulation are unscored. The per-level efficiency factor is e_l = min{1.15, (h_l / a_l)^2} where h_l is the human baseline and a_l the agent's action count, with uncleared levels scoring 0. The public games were released in March 2026, after the base model's February 2026 training cutoff, and an integrity audit scans transcripts for web-tool use or reads of ground-truth sources.

Why This Matters

Impact on research. The paper reframes interaction with an unknown environment as the problem of constructing and validating an executable world model, rather than scaling up reactive policies or answer sampling. It argues for an asymmetric objective: fit to observed transitions is a hard precondition for acting, while goal reachability is pursued only through search. It also reports a counterintuitive practical finding — building a usable world model is simpler than anticipated, whereas inferring the right goal is the harder problem. Twin differs from concurrent ARC-AGI-3 systems by enforcing whole-history replay at the harness level and by hypothesizing the goal before any reward arrives.

Real-world applications:

• Autonomous software and API agents that must infer the semantics of an undocumented system from a few interactions before acting.
• Robotics and industrial control, where a machine could build and verify an executable model of a new piece of equipment before committing costly actions.
• Scientific and clinical experimentation, where hypotheses about hidden state must be tested under strict budget constraints and each probe has a real cost.
• Game and simulation testing, where an agent that recovers the true transition and goal rules could automatically validate levels or detect unintended mechanics.

Industry relevance. The result is directly relevant to companies building agentic products: it shows that a general coding model with a structured harness can match or beat specialized systems on an interactive benchmark, and it quantifies the cost in tokens — 2.60 billion tokens and 91.4 hours of inference for 25 games. The finding that harness enforcement matters more than raw model capability provides a concrete design principle for agent architectures, and the open website replaying all 25 runs action by action and the public GitHub repository support reproducibility.

Future Directions

1. Better goal inference. Goal proposal and temporal state modeling are identified as the main targets for improvement, since goal errors dominate cost and the unfinished games are characterized as accurate dynamics with unresolved goals.

2. Probabilistic twins. Replay validation assumes deterministic dynamics, and exact replay presumes small discrete states — cell equality on 64 × 64 grids is free and decidable, while continuous observations would make every replay an approximate comparison with a threshold to tune. Extending Twin to probabilistic twins is left as future work.

3. Longer temporal context and latent state. step reads a single frame, so mechanics driven by history the grid does not show are out of scope, and truly latent state — a variable no frame ever shows — remains out of scope. The paper notes that bp35's scrolling camera, a partial-observability case, is handled by spending one probe per place.

4. Larger search budgets and stronger priors. Planning and goal discovery run under fixed budgets, so a goal beyond the horizon goes unfound; the authors suggest stronger priors may help agents infer goals as quickly as humans do when playing an unfamiliar game for the first time.

Target Audience

This paper is most useful to AI researchers working on agentic systems, world models, program synthesis, and interactive reasoning benchmarks; to engineers building autonomous agents that must act in undocumented environments; and to benchmark designers interested in how action-efficiency scoring interacts with exploration strategies. Readers already familiar with reinforcement learning and LLM-based agents will get the most from the methodological detail and the ablation, while those focused on evaluation will find the replay-validation and goal-inference measurements the most distinctive part.

Authors’ abstract

We present a Test-time World-model Inference (Twin) system, in which a frontier coding agent writes an executable world model for completing continual learning tasks, such as ARC-AGI-3 games. Traditional approaches hand-engineer such models, one custom design per task. Each game hides its rules and goal, and our system constructs them from simulation and interaction alone. Its inductive prior over grid games is strong enough to recover the true transitions of the game and the goal on nearly all levels. Replay validation happens in a twin world model. The harness enforces that an action is not made until the program reproduces every previous observed game transition. Each mismatch between a world model prediction and the actual action result becomes a counterexample that is used to repair the world model. Twin clears 179 out of 183 levels (97.8%), and does so more efficiently than humans in 158 out of 179 levels (88.3%). The system infers the goal before any reward on 156 of the levels it clears (87.2%), and in the remaining levels automatically discovers the goal by search. The benchmark scores completion and action efficiency, between 0 and 100, against humans playing each game for the first time. Played directly, the base model scores only 7.8%; an off-the-shelf harness increases it to 61.1%, whereas our twin world model increases the same base model to 93.3%, clearing 23 out of 25 games. Building a usable world model is simpler than anticipated, whereas the harder problem is inferring the right goal.