Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement

AI summary

Overview

Research area: Robotics — robot policy learning, code-as-policy, agentic/recursive self-improvement (RSI), and vision-language-action (VLA) models.

Technical level: Advanced. The paper assumes familiarity with VLA models, agent harnesses, world action models, robot proprioception, inverse kinematics, and benchmark evaluation protocols.

One-sentence scope: The paper proposes Code-Only-as-Policy (COAP), which treats the robot and its environment as an "Embodied Turing Machine" whose state is measured and stored explicitly in code, and evaluates a code library developed offline by coding agents on RoboDojo's 42 bimanual tasks, reaching a 70.24% success rate with no model in the control loop at test time.

What This Paper Is About

Most robot policies keep a model in the control loop: a vision-language-action model (VLA) maps observations to actions, or an Agent Harness (such as Agent-as-Policy or Harness VLA) queries a vision-language model (VLM) at run time. This paper argues for a different view: the embodied world is a Turing machine whose tape is the robot and environment state.

The goal is to show that if this state can be represented accurately enough, decision making can be written entirely in code — code that measures and tracks the robot, environment, and task state from camera images and proprioception, runs the same program across every episode of a task, and shares one library across different tasks without a VLM or VLA in the loop.

Key Contributions

1. Code-Only-as-Policy (COAP). The authors model the embodied world as a Turing machine in which robot and environment state can be represented explicitly in code variables. The same code applies across episodes, and different tasks share a library developed offline by a coding agent.

2. An analysis of three advantages of code over VLAs and Agent Harnesses. The paper argues COAP has Explicit State (inspectable, persistent, accurate), Execution (controllable, recoverable, efficient), and Extensibility (reusable, fast to evolve, cumulative), each broken into three properties and compared against baselines in a table.

3. Code as the medium for recursive self-improvement (RSI). Because code iterates efficiently and each change is explicit and controllable, coding agents can develop the library in a closed loop, with each accepted change gated on not regressing other tasks.

4. Code as a data engine. Since the policy is not episode-specific, one program runs on different episodes, and the paper states COAP can generate successful trajectories stably and efficiently as new training data for VLAs and Agent Harnesses.

Main Findings

• COAP reaches 70.24% success rate on RoboDojo's 42 bimanual tasks, which the paper states is 38.9 percentage points above the current state-of-the-art method (31.38%), more than 40 points above the strongest World Action Model (29.64%), and above the strongest VLA (27.96%). Its benchmark progress Score is 75.45, a margin of +39.18 over the best baseline.

• COAP has the highest success rate in each of the five RoboDojo dimensions: Memory 89.9%, Precision 75.1%, Open 64.1%, Generalization 65.7%, and Long-Horizon 56.4%. Margins over the best baseline are largest on Memory (+39.6) and Precision (+41.6), and smallest on Long-Horizon (+13.0).

• Generalization is split between layout types: COAP reaches 77.3% on standard layouts and 54.1% on randomized layouts, for 65.7% overall on Generalization.

• Explicit state helps memory and precision tasks. On RoboDojo Memory tasks, COAP reaches 90%, against 47% for the Agent Harness and 33% for the VLA. On Precision tasks it reaches 75%, against 33% for the Agent Harness and 27% for the VLA.

• Measured state approaches oracle state. The same program succeeds in 63% of episodes using measured state and 83% using oracle state, and the paper reports that each fixed measurement narrows this gap.

• Execution is cheap and model-free. A control step takes a median of 0.3 ms on a CPU, and an episode uses 0.8–2.4% of the compute of a VLA and less than 0.1% of that of an Agent Harness.

• Failure recovery is automatic. In episodes where something went wrong, the program noticed the problem itself in 76% of cases and re-attempted with a new method in 34% of cases. Backtracking matters more as tasks grow longer, since without it one failed step ends the task.

• Code is heavily reused across tasks. 83% of the code that a new task runs comes from the shared library; a new task only reuses, inherits, or extends it.

• New tasks are fast to develop without data or training. On tasks that had no program when the library was frozen, COAP reached an average success of 47% within 15 hours. By comparison, OpenVLA reports 5–15 hours of fine-tuning per task on 8 A100 GPUs, and the strongest VLA on the same tasks reaches 4.7% after training on demonstrations.

• Capabilities accumulate rather than overwrite. Default-off parameters provide a compatibility guarantee: for every task that does not pass the new argument, an extension must leave the execution trace unchanged on the fixtures exercised by unit tests. The paper contrasts this with sequential fine-tuning of π0.5 on five real-world tasks, where the average score drops from 86.9 to 31.4.

• Parallel branching makes development efficient. Development is organized as a breadth-first search over decisions, with each variant a complete runnable program evaluated concurrently. A single change rarely helps, but with more branches a better change is found more often, its gain is larger, and it comes sooner in wall-clock time at the cost of more GPU time. Of 36 branches, few helped; decisions tried with two or more variants found an accepted change in four of five cases.

Methodology in Plain English

The authors frame a robot plus its environment as a Turing machine: the "tape" is the state, and the policy is the fixed rules that read and write it. They then implement that machine as a shared code library.

At each step, the code maintains an explicit state with three parts, each with the same form in every task. The environment state records each object's pose, size, and category, plus spatial relations between objects, and each record stores the error, source, and time of its measurement. The robot state records joint angles, end-effector pose, gripper opening, and reachable poses for each arm, taken directly from proprioception. The task state records the current stage, arm assignment, retry counts, and information the task must retain (for example, the color under each cup); the paper describes this part as the internal state of the machine rather than part of the tape.

Perception is ordinary code built on NumPy and OpenCV. In each frame the robot's mesh is projected into the image using measured joint angles and masked out; pixels differing from the table color form candidate regions; each connected region whose lower edge rests on the table becomes an object candidate. Boundary pixels are converted to table positions by intersecting camera rays with the table plane, and a known 3D object shape is fitted near that position by searching over position and yaw and keeping the best overlap. Camera models, robot meshes, object shapes, and table height are constants in the code because they are published with the benchmark.

The code is layered with dependencies pointing downward: task programs call object-specific code, which calls manipulation primitives (grasp, pick, place, push, pour), which call perception and motion code. Task code specifies what to do; lower layers determine how. New tasks extend the library in one of three ways: composing existing modules, inheriting an existing task and overriding a single decision, or adding a default-off parameter to a shared function.

Improvement happens offline. Coding agents (Claude Opus 5.5; the abstract names "Opus 5.5 Max") work in their own git worktrees, inspect the recorded state of failed development runs to locate the faulty variable or rule, and propose diffs. The resulting branches form a search frontier of runnable code that can be evaluated in parallel under identical conditions. Before simulation, each branch passes two CPU checks: the task code is run against a numeric scene to confirm every stage reaches its target state, and robot and object meshes are checked for inverse-kinematics solutions and collisions at every waypoint. Debugging and selection use separate data: agents debug on a development set they define themselves, while selection compares success counts on a validation split. A branch is accepted only if the targeted task's success count improves by a margin above rerun noise; otherwise the current code stays deployed. The development does not involve a benchmark's test set, and there are fixed rules such as no hacking for single episodes and no oracle state at test time.

Evaluation uses RoboDojo's 42 bimanual tasks spanning Memory, Precision, Open instruction following, Generalization, and Long-Horizon manipulation, with each task evaluated over three seeds of 50 layouts each using the official evaluation client, judges, and summary script. The policy receives only three RGB camera streams, proprioception, and the language instruction — no depth, no segmentation, no model service at test time. Baselines come from the official RoboDojo leaderboard, listing its 23 entries at or above π0.5 grouped into Agent Harness, World Action Model (WAM), and VLA categories.

Why This Matters

Impact on research. The paper argues that code offers properties that learned policies structurally lack: inspectable state, controllable and reversible execution, and cumulative rather than destructive improvement. It frames code as a medium for recursive self-improvement, where every change is tested offline and only non-regressing improvements are merged — a sharp contrast to VLA training, which requires new data and retraining for each task and can degrade earlier tasks. The paper also notes that LIBERO-PRO shows benchmark success can reflect memorized training configurations, and positions code as a way to separate measured state from memorization.

Real-world applications (as implied by the evaluated tasks and claims):

• Bimanual manipulation tasks that require remembering a value after it becomes unobservable, such as remembering colors under cups and uncovering them in a specified order.
• Precision assembly and placement tasks, where the paper reports that millimetre-accuracy tasks can re-measure with a wrist camera before the final approach.
• Instruction-following tasks, including sorting objects into baskets named in the instruction, where a task can inherit an existing flow and override a single decision rule.
• Deployment on cheap CPU-only hardware, since a control step takes a median of 0.3 ms with no model service at test time.

Industry relevance. The cost argument is concrete: an episode uses 0.8–2.4% of the compute of a VLA and less than 0.1% of an Agent Harness, and the robot's behavior is reproducible because no random model is sampled per decision. The paper also positions COAP as a data engine that can generate successful trajectories for training VLAs and Agent Harnesses, which the authors note can benefit the development of those systems.

Future Directions

• Where COAP still fails is only partly addressed. The paper's error analysis section is truncated in the supplied content, so the full enumeration of failure modes is not reported here. The visible text states that current coding ability has limits and that the offline RSI loop needs further work.

• Raising the upper bound of COAP. The authors state the upper bound lies in how accurately the state is represented for decision making and how robust the code logic is. Closing the reported gap between 63% success on measured state and 83% on oracle state is a direct target.

• Generalization remains the stated weakness. The paper acknowledges that COAP needs a small amount of task-specific code logic, though it argues Agent-as-Policy and Harness VLA also involve task-level memories and some rely on episode-level logic. The 54.1% on randomized layouts versus 77.3% on standard layouts marks the size of this gap.

• COAP as a data engine. The paper proposes that trajectories generated by code can serve as training data for VLAs and Agent Harnesses; the specifics of that pipeline are described in a discussion section that is truncated in the supplied content, so the details are not reported here.

Target Audience

This paper is most useful to robotics and embodied-AI researchers working on robot policy design, code-as-policy methods, and agentic self-improvement; to engineers building manipulation systems who need inexpensive, reproducible, model-free execution; and to researchers training VLAs or Agent Harnesses who might use code-only policies as a trajectory data source. It will also interest readers following the broader debate over white-box code versus black-box learned policies. A background in VLA models, robot perception, and benchmark evaluation protocols helps, making it advanced rather than introductory material.

Authors’ abstract

Most robot policies keep a model in the control loop: a VLA maps observations to actions, and an Agent Harness, such as Agent-as-Policy or Harness VLA queries a VLM for decision making at run time. We propose a different view: the embodied world is an Embodied Turing Machine, whose tape is the robot and environment state and rules are the policy. If this state can be represented accurately, the decision making can be written entirely in code. We therefore propose Code-Only-as-Policy (COAP): code measures and tracks the robot, environment, and task state from camera images and proprioception, and makes every decision from it. The same code applies across episodes, and different tasks share one library without a VLM or VLA in the loop. Compared with VLAs and Agent Harnesses, we analyze three advantages of COAP: (i) Explicit State: the state can be stored in code; (ii) Execution: code makes decision making controllable, recovers from failures flexibly, and runs fast and cheaply online; (iii) Extensibility: new tasks reuse, inherit, or extend the shared library, so capabilities can accumulate over tasks. These advantages make COAP a suitable medium for recursive self-improvement (RSI): coding agents develop the library in a closed loop, and each change is explicit and controllable. On RoboDojo's 42 bimanual tasks, the resulting library reaches a success rate of 70.24% without a model at test time. The upper bound of COAP lies in how accurately the state is represented for decision making and how robust the code logic is. We thus propose COAP as a new paradigm for embodied tasks; since it applies across episodes, it can also serve as an efficient data engine for VLAs and Agent Harnesses.