REA: A Routed Embodied Agent Runtime for Asynchronous Brain-Cerebellum Control

AI summary

Overview

Research area: Robotics and embodied AI — specifically the systems problem of combining slow foundation-model "brains" with fast reactive control policies ("cerebella") in a single runtime. Category: cs.RO.

Technical level: Advanced. The core idea is stated simply, but the paper is dense with benchmark protocols, statistical tests and system contracts, so it rewards readers already familiar with vision-language-action (VLA) models and embodied agent pipelines.

Scope in one sentence: The paper proposes REA, a three-level runtime (deliberative System 2, routing System 1, reactive System 0) plus an aligned evaluation protocol, and shows across three simulators and two real robots that routing among policies beats any single policy-generation method without ever making a running policy wait on a foundation model.

What This Paper Is About

Foundation models reason well but are far too slow and expensive to act as fine-grained robot controllers — in the authors' own runs, ASPIRE's repair loop spent a median of 43 minutes per LIBERO task, and a Codex + GPT-6 pipeline burned 89M input tokens to write programs for 60 LIBERO-Pro tasks. Fast policies such as VLAs and whole-body controllers are efficient but generalize poorly: VLAs that exceed 90% on LIBERO collapse on LIBERO-Pro's perturbed test set.

The paper's goal is to make these two kinds of policy cooperate in one runtime without the fast loop ever stalling on the slow brain — and to give the field a common harness where agents and VLAs can be scored on identical episodes, which the authors say does not currently exist.

Key Contributions

1. REA, a three-level runtime with a common contract for agents, VLAs and programmatic policies, evaluated on LIBERO-Pro, BEHAVIOR-1K, RoboTwin 2.0 and real Unitree G1 and AlohaMini2 robots. Every policy exposes the same Run interface and returns control at segment or policy boundaries.
2. An embodied router (System 1) that combines three mechanisms: a route memory of each policy's measured per-task success and cost (so dispatch is a lookup, not a reasoning call), a boundary-triggered judge gate that consults the brain once per policy return to catch silent failures, and a shared-budget escalation ladder that bounds recovery under one step and time budget.
3. An aligned evaluation protocol that runs brains, routers and policies of different families on the same (task, seed) pairs, initial-state semantics, horizon and wall budget, with a single final score.
4. Over 30,000 aligned test episodes verifying the routing and judging contributions, plus a separate 100-episode RoboTwin 2.0 study showing learned VLA policies and Python code policies executing inside the same runtime.

Main Findings

• Routing beats every policy-generation method it draws from. On 3,000 test episodes across all 60 LIBERO-Pro tasks, REA reaches 95.9% success versus 94.7% for the strongest single baseline, Agents SDK + GPT-6 (p = 0.004, paired 95% CI [+0.4, +1.9]%), and comes within 1% of a hindsight per-task oracle at 96.9%. Picking a method per task at random gives 78.2%; picking one method globally gives 94.7%.
• The gain comes from routing, not a bigger pool. On the 37 "aligned" tasks where REA's pool holds exactly the programs the single baselines ran, REA still reaches 95.3% versus 93.7% (54:25, p = 0.001).
• It is also cheaper. REA uses 1,003 steps per success versus 1,343 for Agents SDK (−25%), because each task is sent to its most reliable policy — the SDK program is chosen on only 14 of 60 tasks.
• Route memory is as good as an agent at dispatch, at no model cost. REA-A (agent routing) reaches the same 95.9% (26:26 paired) but is 1.39 s slower per paired episode, of which 0.73 s is waiting for its selection call.
• Waiting on the brain is measurably expensive. A blocking advisor consulted only 1.7 times per episode leaves the control path idle 2.62 s per episode (20% of control time), delays the first action by 1.5 s, and is slower than a single SDK program (13.33 vs 12.04 s) with no success gain. Against it, REA-M is 3.37 s faster (−25%, CI [−31, −20]%) with +1.8% success (90:37, p < 10⁻⁵); against Agents SDK + GPT-6 it is 2.08 s faster (−17%).
• Silent failures dominate. In 590 LIBERO-Pro episodes, an exception-triggered router escalated only 6 times although 166 episodes failed. Adding the judge gate (REA-V) to ASPIRE's Fix policies lifts success from 66.3% to 74.9%, with +5.6% from route memory alone (49:16, p < 10⁻⁴) and a further +3.1% from the judge (37:19, p = 0.022) — at 2.1× the wall time per success.
• Weaker policy libraries benefit more. On zero-shot GPT-6 policies, REA-V lifts success from 19.3% to 26.3% (50:8, p < 10⁻⁴).
• A noisy judge can still help. Its final verdicts are barely more accurate than always answering "done" (0.72 vs 0.67 on the matched subset) or worse (0.67 vs 0.75 on LIBERO-Pro), but its "done" decisions are precise (0.83, 0.905), and by proxy attribution re-runs convert 24% (19%) of failed first runs while undoing 5% (9%) of successful ones.
• The brain is a poor controller but a fine judge. Asked to act directly through 3 cm / 15° Cartesian IK steps, models solve 0 of 600 LIBERO-Pro episodes; a one-token probe solved 1 of 1,987. A completion judge reading scene captions costs 1.3–2.2% success and 7.9–9.6 s per episode on a strong library.
• VLA and code policies share System 0. In the 100-episode RoboTwin 2.0 study (50 tasks, one clean and one randomized scene each), REA reaches 84% with VLA policies and 86% with VLA+code, versus 80% for MEDex, the strongest single VLA baseline. Adding code raises randomized successes from 40/50 to 42/50 with no lost VLA-only successes.
• Transfer holds, with caveats. On BEHAVIOR-1K (five R1Pro pick-up tasks, 50 episodes) REA solves 27/50 versus 25/50 for the Agents SDK agent (7:5, p = 0.77 — not significant, and the margin comes from one task, "camera", 5 vs 1), 18 each for Codex and ASPIRE-style Fix, and 3 for Show-Harness. In the separate 200-scene RoboTwin study REA solves 97 versus Codex CLI's 75 (42:20, p = 0.007); a later-night rerun gave 133 versus 105, 103 and 12, but the same configuration moved 36 scenes between nights, so only the side-by-side pair is treated as window-free.
• Real robots. On a Unitree G1 walk-and-push-chair task REA succeeds in 8/10 trials versus 5 for Agents SDK, 4 for Codex and 2 for ASPIRE (p = 0.023). On AlohaMini2 block placement REA completes 10/10 versus 8/10 for ASPIRE, 5/10 for Codex and 4/10 for Agents SDK; in a separate 20-trial study routing asynchronously between π₀.₅ and a Code controller gives 8/10 versus 3/10 for π₀.₅ alone, with placement failures falling from five to zero.

Methodology in Plain English

The system is built around one rule: no running policy ever waits for the brain. Everything else follows from that.

Three levels. System 0 holds the fast policies — frozen Python programs on LIBERO-Pro, learned VLA action-chunk policies on RoboTwin, whole-body controllers on hardware. Each runs its own closed loop at native rate and hands control back at natural boundaries (an action chunk, a motion interval, or a program pausing every 100 control steps with its stack preserved). System 2 is the foundation-model agent; it can write policies offline and can judge or advise online. System 1 is a router that emits no actions at all — it only decides which policy runs next and when, if ever, to consult System 2.

Route memory replaces in-loop reasoning. Before evaluation, the router runs every candidate policy on a handful of development episodes (seeds disjoint from test seeds) and records its success rate and mean step count. At test time, choosing a policy is an argmax lookup over that table, with ties broken by lower cost. Ten development seeds per program (1,790 episodes total, zero model tokens) is the entire routing cost, versus 1,249–1,328 development episodes and 29–89M input tokens for the generation pipelines.

The judge gate fires on returns, not errors. Because most failures raise no exception at all, REA-V calls a vision-language judge once each time a policy returns, comparing the episode's initial frame with the current one. "Done" ends the episode and protects completed work; "not done" escalates. A free exception gate that only fires on exceptions or truncation is kept as an alternative, and the paper recommends choosing between them per policy library on development episodes.

Escalation is budgeted, not open-ended. A ladder of candidate policies launches each retry from the current state, and every policy run and every brain call draws from the same remaining step horizon and wall-clock budget — so an episode never exceeds what a single policy would have been allowed.

For the RoboTwin VLA study, the same router interface instead hosts a learned selector: an encoder maps the initial instruction, images and robot state to a representation, a success head scores each candidate, and the top-scoring policy is fixed for the episode. Adding a code-policy head extends the seven-VLA selector to a mixed pool without touching the execution interface.

Comparison discipline. Within each matched comparison, every method uses the same (task, seed) pairs, initial state, horizon (4,000 steps), wall budget (900 s) and, where shared, byte-identical policy code. Success differences use paired exact McNemar tests with paired 95% intervals and task-clustered bootstrap for time; the headline contrasts survive a Holm adjustment.

Why This Matters

For research: The paper's sharpest claim is infrastructural. Agent-based systems and VLA-based systems are normally built on their own simulators, observation formats and success criteria, which makes cross-family comparison nearly impossible. REA offers a shared contract and a worked example of running code policies, seven VLAs and a real humanoid through one interface on identical episodes — plus a negative result (models as direct Cartesian controllers: 0 of 600 episodes) that reframes where the brain belongs in the stack.

Real-world applications:

• Humanoid and mobile manipulation — the Unitree G1 walk-and-push-chair result shows a deliberative model plus a reflex controller handling a task that neither layer solves alone.
• Bimanual tabletop assembly and pick-and-place — the RoboTwin 2.0 mixed VLA+code pool reaching 86% versus 80% for the best single VLA suggests deployment stacks should be heterogeneous rather than monolithic.
• Long-horizon household tasks — the BEHAVIOR-1K pick-up tasks exercise the escalation ladder over horizons where silent failure is the norm.
• Cost-controlled fleet operation — route memory dispatching with zero model tokens per episode is directly relevant to operating many robots under a fixed inference budget.

Industry relevance: The blocking-advisor measurement (2.62 s idle per episode, 20% of control time, with no success gain) is a concrete argument against putting an LLM in the dispatch path. The finding that a judge's raw accuracy can be near or below a trivial "always done" baseline yet still improve task success is useful for teams tempted to gate deployments on judge accuracy alone.

Future Directions

• Learned routing versus lookup at scale. The paper shows route memory matches an online agent at dispatch and that a learned selector works for VLAs, but the learned selector study only covers initial policy choice; recovery-gate effects were evaluated separately on LIBERO-Pro. Combining learned selection with judge-gated recovery is left open.
• Better judges, and knowing when to pay for one. The judge's "done" precision is what matters and its overall accuracy is

Authors’ abstract

Foundation models are becoming remarkably reliable at high-level reasoning, yet they remain too slow and costly to serve as fine-grained controllers. Conversely, fast policies like VLAs and whole-body controllers offer more efficient reflexes, but struggle to generalize. While pairing a deliberative brain (foundation models) with a reactive cerebellum (fast policies) seems natural for embodied systems, two obstacles remain: methods are built and scored on incompatible stacks, and their asynchronous coordination is difficult. For example, an agent that calls the VLA in the agentic loop can stall the fast execution, while a VLA that calls the agent on errors tends to re-trigger fallbacks repeatedly or completely miss silent failures. To address this, we propose REA (Routed Embodied Agent), a unified runtime where a deliberative System 2, an embodied router System 1 and reactive System 0 policies cooperate under a single golden rule - no running policy waits on the brain. Instead of querying the brain at each failure, REA employs a route memory that stores each policy's measured success ahead of time, a boundary-triggered judge gate that consults the brain on policy execution, and a shared-budget escalation ladder for bounded recovery. Evaluated on 3,000 test episodes across 60 LIBERO-Pro tasks, REA reaches 95.9% success and beats the best single-agent baseline in its own policy pool (94.7%) while reducing control steps by 25% and control time by 17%. In a separate 100-episode RoboTwin 2.0 study, REA achieves 84% success with VLA policies and 86% with VLA+code, compared with 80% for the strongest single VLA baseline MEDex. These results demonstrate learned and programmatic policies operating within the same runtime. REA also transfers to BEHAVIOR-1K and a real Unitree G1 humanoid, achieving the highest observed success among the compared methods.