PyRUA-REAL: Robot Exploration and Adaptive Learning through RSI in the Real World

AI summary

Overview

Research area: Robotics + LLM coding agents — specifically "embodied recursive self-improvement" (RSI), where a coding agent improves robot behavior through real physical trial, code revision, and memory accumulation, with model weights frozen.

Technical level: Advanced. It assumes familiarity with code-as-policy robot control, agent harnesses, and manipulation benchmarking.

Scope: An empirical study of how a two-agent (researcher + engineer) coding system turns real-world manipulation experience into persistent memory, reusable skills, and frozen executable robot programs, evaluated on 8 physical dual-arm Franka tasks and 42 RoboDojo simulation tasks.

What This Paper Is About

Large-model coding agents can write robot programs, but a single success does not make a capability. The paper asks how an agent can accumulate physical interaction experience — successes, failures, and diagnostics — into memory and reusable code so that later attempts are faster, more reliable, and transferable to new tasks. The authors build PyRUA-REAL to study this loop end-to-end in the real world: exploration, code refinement, skill retention, and eventual deployment of frozen programs that run without any online model calls.

Key Contributions

1. An empirical study of skill acquisition and improvement beyond first success. Across eight real-world manipulation tasks on a dual-arm Franka platform, the authors trace how coding agents convert exploration into executable skills and keep improving them after the first successful rollout, documenting gains and trade-offs in completion, execution time, and placement precision across program revisions.

2. Diagnostic case studies on the factors shaping embodied RSI. Comparative experiments examine agent coordination (with versus without a reviewer agent), the richness of execution feedback, the effect of explicit task specifications, and transfer from a previously acquired primitive-skill library.

3. Evaluation of deployed capability and its cost. Frozen programs are compared against online-agent execution modes, quantifying the execution-time cost of per-step agent decisions and identifying tasks that still require online assistance.

4. A structured persistent-experience architecture. Experience is organized into three levels — L0 facts (robot/workspace conditions), L1 memory (problems, attempts, outcomes, failure causes), and L2 implementation (task programs and reusable primitives) — with retrieval split by agent role.

Main Findings

• Real-world capability acquisition: Agent-free programs were produced for seven of eight physical tasks. Approximate time to first success ranged from 1.5 hours (block storage) to 45 hours (Push T). Recorded development success across the seven tasks ranged from 91.67% to 100%. The abstract reports an average success rate of 91.1% across the eight real-world tasks; the paper notes these rates aggregate attempts across code revisions rather than measuring a single frozen program.

• One task resisted full autonomy: Ethernet insertion remained unsuccessful without online agent participation. The development record contains 0/4 successes with scripted execution versus 10/17 with a workflow using GPT-6-Astra Medium for online decisions between stages; in one successful run, motion took 281.8 s and inter-stage checks/decision gaps took 220.8 s, 44% of the 502.7 s total.

• Continued improvement after first success: For five-block stacking, median stack–unstack cycle time fell from 285.3 to 156.0 s, mean visually estimated lateral error from 2.69 to 0.44 mm, and mean absolute angular error from 1.37 to 0.25°. Full-cycle success rose from 88.7% to 98.7% across two reported smooth-stage evaluations. The authors state these errors are visual estimates, not independent measurements of physical accuracy.

• Strategy changes can outperform local tuning: Tic-tac-toe success rose from 50% in the first version to 100% in the next (with later regressions before returning to 100%), and game-and-reset time fell from 577 to 535 s overall, most of it between V4 and V5 (573 to 536 s) when a revision replaced sequences of elevated free-space waypoints with a single JointMotion.

• Optimization priorities shift over time: Hanoi first prioritized reliability (median cycle time reaching 1402 s in the robust stage) then speed (883 s and 793 s in two speed-focused versions), with full-cycle success rising from 60% initially to 100% in the final version.

• An extra reviewer agent slowed exploration: Median result-to-next-trial interval was 16.9 minutes with a reviewer versus 3.4 minutes without, across 40 and 9 recorded transitions respectively. Six submission–rejection–revision cycles in Hanoi took 5.1–19.0 minutes each (median 8.5 minutes). The reviewed phase recorded no verified disk-on-peg insertion over 45.0 hours; the subsequent direct-testing phase reached its first insertion 2.8 hours after the first grasp test, though the authors caution the phases are sequential so this does not isolate the reviewer's effect.

• Task-level reassessment breaks local deadlocks: In all three physical reassessment cases the next trial succeeded, with subsequent records of 3/3 for stacking, 1/1 for towel folding, and 3/3 for Hanoi. Simulation results were less uniform: 11/17 for Push T after reassessment, 4/11 for Imitate sorting sequence.

• Richer feedback helps, at reduced optimization time: On four RoboDojo tasks (five revisions, 18 test executions across six held-out layouts), mean success was 15.3% for basic outcome reports, 45.8% for numeric state reports (with privileged ground-truth states), and 54.2% for video plus joint trajectories. BVJ also used less optimizer time than B (25.4 versus 31.8 minutes) with slightly higher token usage (3.65 versus 3.54 million).

• Explicit task specifications improve robustness: Providing workspace boundaries and target position/orientation ranges plus continued revision raised mean success in all five RoboDojo categories by 11.4–36.6 percentage points. Open tasks went from 14.4% to 51.0% and Memory from 53.8% to 65.2%; Stack bowls went from 0.0% to 86.4% and Fasten screws from 9.1% to 81.8%. The gain was not universal — Swap blocks declined from 68.2% to 59.1%. This comparison includes extra exploration, so it does not isolate the specifications alone.

• Positive but uneven skill transfer: With a primitive-skill library from earlier tasks, agents reached 40.0% mean success versus 21.3% without it; five of eight tasks improved (classify by language and play stacking toy each +50 points, Imitate sorting sequence +30, general pickup and make toast each +10), while fill egg holder, push T, and swap blocks showed no gain.

• Frozen programs are much faster: Offline-coded programs for tic-tac-toe, five-block stacking, and Tower of Hanoi completed with median successful cycle times of 8.99, 2.99, and 16.72 minutes. Online coding (PyRUA-Lean) took roughly 4.6–5.0 times as long on tic-tac-toe and stacking, and online action selection (GPT-6-Astra) roughly 9.9–11.5 times as long. Agent thinking accounted for 63–90% of recorded online execution time, and neither online baseline completed Hanoi within 120 minutes.

• RoboDojo comparison, with an internal inconsistency: The abstract states 84.4% average success; Section 4.3.3 reports 84.42% capability-averaged success across 42 tasks versus 50.35% for PyRUA-Lean and 22.48% for GPT-6-Astra; the Introduction states 84.82%. By group, Precision was 74.00% versus 38.25%, Long-Horizon 56.00% versus 43.75%, and Generalization slightly lower at 41.17% versus 43.17%. The authors note these are reported results from separate evaluations with different or unspecified seeds.

Methodology in Plain English

The system splits labor between two LLM-based agents with fixed model parameters. A researcher decides what to try next at the task level: it sets an objective, supplies relevant facts, past approaches and skills, and defines when to stop. An engineer writes the code, runs it on the robot, diagnoses failures, and iterates locally. They communicate through structured units called experiments: the engineer returns a report containing results, findings, evidence, and code, and the researcher decides whether to refine, change direction, or move on.

Improvements persist in a three-level repository — environment facts (L0), narrative memory of problems and failure causes (L1), and actual code including reusable primitives (L2). The researcher curates this repository from engineer reports and credits a skill as "used" only when it appears in an experiment's final implementation.

The robot stack is two Franka Research 3 arms with Robotiq-85 grippers, a head-mounted ZED Mini, and wrist-mounted Intel RealSense D405 cameras. Platform calibration and task-specific controllers were not given in advance and had to be discovered during exploration. Frozen programs may call locally deployed perception models such as SigLIP and SAM, but make no runtime language-model calls — they still react to live observations through sensing, conditionals, and recovery logic.

The empirical work mixes real development histories (success rate, cycle time, and placement error tracked across revisions), controlled comparisons (with/without reviewer; three feedback interfaces; with/without task specifications; with/without skill library), and RoboDojo simulation benchmarks. Several studies use a single development session per condition, which the authors flag as leaving variability unmeasured.

Why This Matters

Impact on research: The paper reframes robot learning from "train a policy" to "let an agent accumulate executable experience," and gives concrete, if partly confounded, evidence on where the improvement comes from: motion-strategy changes, richer feedback, explicit operating-condition specs, and task-level reassessment rather than endless local patching. The finding that a reviewer agent adds 13.5 minutes of median latency per trial and that a reviewed phase produced no verified insertion over 45.0 hours is a directly useful negative result for anyone building multi-agent coding loops.

Real-world applications:

• Warehouse and logistics manipulation — bin picking, block sorting, and storage of scattered items into containers.
• Assembly and cable/connector tasks — Ethernet insertion illustrates both the value and the current limits of frozen programs in contact-rich, high-precision work.
• Household and service robotics — towel folding with bimanual coordination, and long-horizon multi-step routines.
• Industrial deployment pipelines — freezing explored programs into fast, latency-free executables that still respond to sensor feedback, avoiding per-step model inference costs.

Industry relevance: The 4.6–11.5× execution-speed gap between frozen programs and online agent control, with agent thinking consuming 63–90% of online time, is a direct argument for the develop-once-deploy-cheaply pattern. The skill-library transfer results (40.0% versus 21.3%) and the failure of autonomous scripted execution on Ethernet insertion (0/4 versus 10/17 assisted) map out where human-in-the-loop or online-agent fallback still pays for itself.

Future Directions

• Escape method-selection deadlocks. The truncated discussion begins to attribute stalls to the set of methods an agent considers and to geometric localization errors; systematically expanding the method repertoire an agent searches over, and detecting when a stall is a method problem rather than an implementation bug, is a clear next step.

• Decide when online assistance is required. Ethernet insertion succeeded 10/17 only with online decisions between stages, and 44% of one successful run was inter-stage checking. Learning an explicit criterion for when to escalate from frozen program to agent-in-the-loop — rather than discovering it task by task — remains open.

• Separate the effects of the studied factors. The reviewer comparison, the task-specification comparison, and the reassessment analysis are each confounded by sequencing or additional exploration, and the feedback study uses one development session per task–condition pair. Replicated, isolated ablations would turn the qualitative guidance into quantified design rules.

• Scale and standardize evaluation. Results are reported as development-record aggregates rather than repeat tests of frozen programs, and the RoboDojo numbers come from separate evaluations with different or unspecified seeds. Independent repeat testing and a consistent evaluation protocol (the authors cite an October 24, 2026 comparison snapshot) would make cross-method claims firmer.

Target Audience

Researchers and engineers working on LLM-driven robot control, code-as-policy systems, and agent self-improvement; robotics teams evaluating whether to deploy frozen agent-written programs versus online agent inference; and practitioners building multi-agent coding harnesses who want evidence on reviewer overhead, feedback channels, and skill reuse. Readers focused on low-level control or classical manipulation learning will find little here — the contribution is about the improvement loop, not the controller.

Authors’ abstract

Coding agents powered by models such as GPT-6-Astra connect spatial reasoning with executable robot behavior, raising the question of how such agents can produce lasting improvements in robot capabilities from physical interaction. In this work, we study embodied recursive self-improvement (RSI) through real-world exploration, code refinement, and deployment using PyRUA-REAL, a framework in which coding agents retain interaction experience as persistent memory and reusable executable skills that inform subsequent exploration and improvement. Our empirical study places physical manipulation at the center of the improvement process, examining how agents collaborate, what kinds of execution feedback support effective code refinement, and how task specifications and previously acquired skills influence robustness and task transfer. We further investigate the efficiency and capability of the resulting programs when deployed without involving an online agent for decision-making. Across eight real-world tasks, PyRUA-REAL achieves an average success rate of 91.1%. In complementary evaluation on 42 tasks from the RoboDojo benchmark, it achieves an average success rate of 84.4%. This study provides empirical guidance for turning physical exploration into reusable robot skills that support continued improvement and robust, efficient deployment.