TinyAstral: Transferring Self-Evolving Harnesses to Tiny Generalist Manipulation Agents

AI summary

Overview

Research area: Robotics (cs.RO) — vision–language-model (VLM) agents for robot manipulation, combining agent-harness evolution with knowledge distillation.

Technical level: Advanced. The paper assumes familiarity with VLM-based robot agents, tool-calling planners, recursive self-improvement, and supervised fine-tuning of language models.

Scope (one sentence): The paper shows that improving the harness (tools, feedback, memory, recovery logic) around a frozen large teacher planner, then distilling that evolved system into 0.8B–4B students, produces tiny manipulation agents that stay close to their teacher, generalize to unseen benchmarks, and cost far less inference compute.

What This Paper Is About

A robot agent built on a VLM depends on two things: the planner model that decides what to do, and the "harness" around it — tool interfaces, execution feedback, task memory, and failure recovery — that determines how actions are carried out. The authors ask whether improvements made to a large teacher's harness can be carried into a much smaller agent, so that the same decision-making quality runs at far lower inference cost. They answer with TinyAstral, a two-stage recipe: evolve the teacher's harness while its weights stay frozen, then distill the teacher's reasoning and recovery behavior into compact 0.8B–4B students that occupy the same frozen harness at deployment.

Key Contributions

1. A two-stage harness-evolution-then-distillation pipeline. Stage 1 improves a frozen teacher's manipulation harness through master-guided recursive self-improvement (RSI); Stage 2 freezes that harness, collects teacher trajectories containing reasoning, tool calls, and injected-failure recovery, and trains compact students to replace the teacher inside the same harness.
2. Evidence that harness gains transfer. The evolved harness improves other frozen planners without retraining (DeepSeek-V4.1-Flash +30.6 pp, untrained Qwen3.5-4B +26.6 pp), and students distilled at four successive harness versions track their corresponding teachers.
3. Compact agents with lower inference compute cost. A single TinyAstral-4B model is evaluated across six benchmarks, outperforms every open competitor the authors ran on identical episodes, and uses about one fifth of its teacher's accelerator time per episode in the deployment measurement.
4. A statistical-selection protocol for harness edits. Every candidate edit is accepted only if it passes a pre-registered paired A/B test on disjoint selection seeds, with automated code audits enforcing that edits are general (no branching on task names) and behave step-for-step like the parent when their trigger does not fire.

Main Findings

• Harness evolution substantially lifts a frozen teacher. On the same 500 RoboTwin episodes, the teacher (Qwen3.8-Flash-Next) rises from 33.4% strict success under harness H₁ to 63.6% under the final harness H*.
• Distillation carries the gains into tiny models. Students distilled at four harness versions score 36.2%, 49.8%, 60.0%, and 62.8% — each close to its corresponding teacher (33.4%, 49.4%, 60.4%, 63.6%). Within the final harness, the 4B student beats the untrained 4B planner by 21.6 pp (62.8% vs. 41.2%).
• The harness is reusable across planners. DeepSeek-V4.1-Flash improves from 32.8% to 63.4% (+30.6 pp) and the untrained Qwen3.5-4B from 14.6% to 41.2% (+26.6 pp) under the same harness lineage, with fixed weights.
• The 4B student stays close to its teacher on all three training benchmarks. 62.8% vs. 63.6% on RoboTwin, 93.6% vs. 94.7% on Guava, and 78.3% vs. 80.0% on LIBERO-PRO.
• It beats open competitors on identical episodes. 93.6% vs. 84.4% for Guava-4B; 78.3% vs. 70.3% for SenseNova-RoboRSI-27B on LIBERO-PRO; 12.8% for PyRUA-Lean-4B on RoboTwin. It also beats the authors' own teacher model placed inside competitors' harnesses: 93.6% vs. 78.9% (Guava harness), and 78.3% vs. 57.3% (MotorMind) and 53.0% (PyRUA-Lean) on LIBERO-PRO.
• Gains hold on three benchmarks never used for RSI or training. On SimplerEnv, VLABench, and EB-Manipulation the 4B student is within 5.0 pp of its teacher and within 2.4 pp of the 552B DeepSeek-V4.1-Flash running the same harness; it also exceeds published π₀-FAST results on both SimplerEnv robots and π₀ on VLABench, although those baselines were fine-tuned in-domain.
• Teacher reasoning is the most important distillation ingredient. Blanking the reasoning text while keeping all samples identical drops LIBERO-PRO by 19.3 pp, SimplerEnv Google VM by 11.2 pp, and VLABench by 10.0 pp; on LIBERO-PRO, 49 of 300 episodes then end at the call limit versus 7 under the full recipe.
• Auxiliary real-robot data mainly improves offline real-observation decisions. Next-call accuracy on held-out real-robot episodes rises from 0.235 to 0.923 with no statistically detectable change in simulated success.
• Smaller students are competitive but uneven. TinyAstral-0.8B reaches 61.8% on RoboTwin and 93.8% on Guava, close to the 4B student, but drops to 54.7% on LIBERO-PRO versus 78.3%; on the zero-shot benchmarks the 0.8B and 2B students stay within 3.0 pp of the 4B model.
• Deployment cost is much lower. On 20 shared SimplerEnv episodes the student uses 25.3 accelerator·seconds versus 123.8 for the teacher (4.9×) and 249.6 for DeepSeek-V4.1-Flash (9.9×), with the shortest median episode (25.3 s vs. 31.0 s and 31.2 s).
• The student recovers from failures rather than stalling. Of 271 zero-shot episodes the student solved but the untrained 4B model did not (same harness), 171 (63%) end with the untrained model exhausting its call budget.
• It closes the loop on real hardware. The same student runs on an SO-ARM101 arm, sorting two balls into matching boxes and stacking three tape rolls.

Methodology in Plain English

The system has two stages, and model weights never change in the first one.

Stage 1 — evolve the harness. The teacher planner (Qwen3.8-Flash-Next) is frozen. A "master" consisting of frontier LLM agents reads the teacher's logged trajectories — images, tool calls, tool results, and reasoning — and sorts each failure into one of ten failure classes (off-target placement, false completion, missing primitives, task-memory errors, instance misidentification, and so on). The master then writes candidate harness edits: new tools, more honest tool returns, perception fixes, failure handling, or task-memory corrections. Humans fix the rules and budget in advance. Candidates are tested against the current harness on the same task/seed pairs, with training, selection, and evaluation seeds kept disjoint; an edit is accepted only if a pre-registered paired statistical test shows improvement on targeted or triggered pairs without regressions elsewhere. Guardrails forbid task-name branching, require byte-identical prompts and tool descriptions to the parent, and require each new component to fire through an explicit trigger whose absence is verified to leave behavior step-for-step unchanged. The final integration phase accepted three of seven candidates, and the lineage of accepted snapshots runs from H₀ through H₁₆ to the final H*.

Stage 2 — distill into a small student. With H* frozen, the teacher collects trajectories. Crucially, the training targets keep the teacher's <think> rationale together with the next tool call, so the student learns why as well as what. Because successful episodes rarely show recovery, the authors build "recovery branches": they replay the first k calls of a successful episode with recorded results, inject a perturbation (object slips, target moves 6–10 cm, grasp misses, push falls short, placed object displaced), let the teacher continue online, and keep the branch only if the perturbation took effect and the teacher still reached strict success. On RoboTwin these branches are subsampled to 20% (3,471 branch and 13,857 main-episode samples). Only strict successes are kept — episodes the agent ended with an explicit done call that the benchmark verified — and success labels never enter the training messages. Offline real-robot data from public datasets (AgiBot World plus several Open X-Embodiment sources) is recast as three auxiliary tasks (27,918 state/failure-judgment, 4,187 sub-goal-planning, 9,771 next-tool-call samples), with the teacher supplying reasoning via a STaR-style rationalization step. The final training set has 83,753 decision samples, half simulated; Qwen3.5-4B is fine-tuned for 2 epochs and deployed in bf16 on one accelerator inside the frozen harness.

Evaluation. Six benchmarks: three tied to training data (RoboTwin-50, 500 episodes; Guava, 450; LIBERO-PRO, 300) and three untouched by RSI or training (SimplerEnv, 960 episodes across Google Robot and WidowX; VLABench, 500 episodes; EB-Manipulation, 228). The primary metric is strict success, which counts an episode only if the agent voluntarily declared completion and the final state satisfies the benchmark. The environment's success flag never enters the model's input.

Why This Matters

The paper reframes where capability in robot agents comes from: much of the improvement that looks like a "bigger model" problem can be captured in the surrounding harness and then transferred into a small model. That makes high-quality manipulation agents cheap enough to deploy, and it gives a principled, statistically gated procedure for improving a robot system without retraining the planner.

Real-world applications:

• Low-cost robotic arms in warehouses and labs — a 0.8B–4B planner can run on a single accelerator, matching the compute profile of edge deployments rather than datacenter inference.
• Household and service robots — the demonstrated SO-ARM101 tasks (sorting balls into matching boxes, stacking tape rolls) and the recovery behavior under injected perturbations point at unstructured settings where grasps fail routinely.
• Flexible manufacturing and kitting — the harness is planner-agnostic, so a site can swap in whatever frozen VLM is available or approved and keep the same tools, memory, and recovery logic.
• Fleet-wide maintenance — because the harness improves independently of weights and other planners inherit its gains without retraining, harness updates can be rolled out across a heterogeneous robot fleet.

Industry relevance: the compute result (25.3 vs. 123.8 accelerator·seconds per episode) changes the unit economics of always-on robot inference, and the "evolve the system, freeze the model" pattern is directly applicable to any company building tool-using agents around a third-party or locally hosted model it cannot or does not want to retrain.

Future Directions

• Extending RSI to more environments. The authors state this explicitly as future work; currently harness evolution is developed only on RoboTwin, and task memory is used only there (the other five benchmarks use none), so how far the loop generalizes to new domains is untested.
• Closing the capacity gap on harder, perturbation-heavy benchmarks. The 0.8B student matches the 4B student on RoboTwin and Guava but falls to 54.7% vs. 78.3% on LIBERO-PRO, so where the size floor actually lies — and whether better data or recipe design moves it — remains open.
• Making real-robot supervision count in closed-loop success. Real-robot data lifts offline next-call accuracy from 0.235 to 0.923 but shows no detectable change in simulated success; converting that offline competence into measured closed-loop gains is an open problem.
• Deciding when harness evolution should stop. The loop runs to H* under a human-set budget and accepts edits by paired significance tests; how to allocate that budget, when to stop, and whether evolved harnesses compose across robot embodiments are unanswered questions.

Target Audience

Robotics and embodied-AI researchers working on VLM-based manipulation agents; practitioners building tool-using agents who want a template for improving a system around a frozen model rather than retraining it; distillation and self-improvement researchers interested in transferring scaffold-level gains into small models; and engineering teams evaluating whether small on-device planners can replace large hosted ones for real robot deployments. Readers without background in robot learning will find the high-level argument accessible, but the benchmark taxonomy and statistical acceptance protocol assume an intermediate-to-advanced background.

Authors’ abstract

Robot agents built on vision-language models depend on both the planner and its harness: the tools, feedback, memory, and recovery logic around it. Can improvements to a large teacher's harness be carried into a compact agent with lower inference compute cost? We present TinyAstral, a two-stage approach. We first freeze an open-weight teacher and improve its harness through recursive self-improvement (RSI): frontier LLM agents read the teacher's trajectories and propose harness edits, and each edit is accepted only if it passes a paired statistical test. We then freeze the final harness, collect teacher trajectories with reasoning and failure recovery, and distill a small student that replaces the teacher in the same harness. On RoboTwin, harness evolution raises the teacher's success from 33.4% to 63.6%; the final 4B student reaches 62.8%. Across four harness versions, distilled 4B students score close to their corresponding teachers, and later versions yield stronger students, showing that harness evolution gains carry into compact models. The harness gains also transfer to other frozen planners without retraining. On identical episodes, the final 4B student outperforms open 4B agents, the open 27B harness-RSI model SenseNova-RoboRSI, and our own teacher model placed in three competitors' harnesses. It stays close to its teacher on all three training benchmarks, and on three benchmarks never used for RSI or training it scores close to a 552B planner running the same harness. On the measured deployment episodes, it uses about one fifth of its teacher's accelerator time.