The Trinity:Capability-Driven Recursive Self-Improvement for Multimodal Agents

AI summary

Overview

Research area: Artificial intelligence — specifically recursive self-improvement (RSI) of agent "Harnesses" (the orchestration layer around a frozen foundation model) and agent–environment co-evolution for multimodal agents.

Technical level: Advanced. The paper is written in formal notation (module registries, partial-union operators, trie-based tree construction) and assumes familiarity with agent scaffolds, benchmark suites, and LLM-based evaluation pipelines.

Scope (one sentence): The paper proposes a framework in which a three-role loop (Creator, Explorer, Evolver) reorganizes 44 heterogeneous multimodal benchmarks into a shared capability tree, generates simplified verifiable training worlds from it, and evolves a matching tree of specialized Harnesses that are selected at test time by a capability-aware router.

What This Paper Is About

Multimodal agents are limited less by the frozen foundation model than by their Harness — the perception, memory, inference, planning, orchestration and loop logic wrapped around the model — and that Harness is normally improved by hand, which cannot keep up with growing task diversity. Building large, heterogeneous, verifiable training environments by hand is similarly expensive, and existing RSI work evolves Harnesses per task or per benchmark family, which overfits and generalizes poorly across modalities.

The paper's central claim is that tasks with different surface forms (for example ARC-AGI-3 and Micromouse) can share the same underlying capability requirements — accurate perception and state tracking — so those requirements, rather than benchmark labels, should organize both environment generation and Harness evolution. The goal is an automated loop that turns verified interaction feedback into progressively better, capability-specialized Harnesses that transfer to unseen benchmarks and to real robots.

Key Contributions

1. Capability-driven multimodal RSI. Shared capability requirements become the organizing principle of Harness evolution. "Capability-Driven Real2Sim2Real" converts those requirements into compact, verifiable synthetic worlds, producing a reusable module-based Harness tree whose specialization motivates capability-aware routing.

2. Capability–module priors for Harness design. The authors connect task-level capability requirements to reusable module configurations, using measured capability–protocol correlations plus controlled add/remove interventions to test which module support matters for a given task.

3. Efficient evolution and transfer. Targeting shared decision logic yields Harness improvement at low evolution cost, with transfer demonstrated across heterogeneous tasks, unseen benchmarks, and real-world embodied settings.

4. A three-role architecture with recursive accumulation. Evolver re-organizes the task tree into a hierarchical capability tree from verified trajectories; Creator builds and evolves synthetic worlds and consolidates a new Base Harness each iteration; Explorer evolves reusable modules through selective inheritance in a fixed-topology Harness tree, and a Capability-Aware Harness Router picks a compatible branch using only public task information and verified experience.

Main Findings

• Capability organization beats task specialization. Capability Tree (Sim) reaches the best module-only scores of 56.95 Train / 47.26 Unseen, versus 53.04 / 42.92 for a single Global Harness, 51.17 / 40.82 for 15 Task-Domain Harnesses, and 49.25 / 39.43 for six flat Capability-Domain groups. The flat grouping is the cheapest to train (136.94M training tokens) yet underperforms Global.

• Simulation adds transfer on top of the tree. Capability Tree (Real) gains 4.19 / 2.88 points over the flat capability organization; substituting verified synthetic worlds adds another 3.51 / 4.95, while also yielding the lowest inference costs (514.42M Train / 549.07M Unseen). First-iteration training cost is 405.70M for Sim versus 211.56M for Real and 278.25M for Task-Domain.

• Modules and skills are complementary. Base scores 39.06 / 37.45; Module Only 56.95 / 47.26; Skill Only 58.86 / 48.67; combining both at Iteration 1 gives 63.80 / 53.53.

• RSI keeps improving after the first update. From Base to Iteration 4, Train rises from 39.1 to 73.2 and Unseen from 42.1 to 64.1; Iterations 2–4 add a further 9.4 Train and 6.4 Unseen points. Native Train improves from 64.4 to 68.8 after Iteration 1 with smaller later gains.

• Progress is uneven across benchmarks. Between Iterations 1 and 4, ARC-AGI-3 rises 41.0 → 53.6, SciGym 45.2 → 57.1, BuildArena 93.3 → 97.8; Mastermind stays at 88.9 and Eleusis declines 82.2 → 81.1. The final Unseen mean exceeds the Codex (47.5) and DeepSeek (46.2) references, though Crafter and DiG-bench remain below both.

• Agent routing closes most of the gap to an oracle. Random routing scores 52.74 / 46.23; self-routing with public task information alone 65.79 / 52.36; adding capability definitions 70.34 / 58.27; adding retrieved execution experience (the full router) 71.27 / 60.96; Oracle 73.16 / 63.61. The full router also uses fewer estimated tokens (497.89M / 541.28M) than Random (567.08M / 594.25M).

• Capability–protocol associations are structured. Across 29 modules at Iteration 4, the strongest positive links are hypothesis/counterpart inference to Inference (0.744), goal commitment and value trade-off to Loop (0.444), video evidence acquisition to Perception (0.347), perception and state tracking to Memory (0.306), constraint checking and feedback repair to Planning (0.274), and video comprehension equally to Perception and Memory (0.286). The most negative are goal commitment to Perception (−0.361) and perception/state tracking to Loop (−0.313).

• Interventions confirm the priors. Adding a relevant module to the Base Harness raises the score from 39.06 to 58.45 (+19.39), versus 40.08 (+1.02) for an irrelevant module. Removing one relevant installed module from the Iteration-4 Harness drops 73.16 → 70.42 (−2.74) and raises tokens from 496.70M to 517.26M.

• Capability-evolved Harnesses transfer to robots. On SOON, success rises from 0 → 80% in simulation and 0 → 60% on the real robot; on CaP-X, from 10 → 60% in simulation and 0 → 60% on the real robot. Both robots already reach 20% with H^(1); SOON hits its final real score at H^(3), while CaP-X improves from 40% to 60% between H^(3) and H^(4).

Methodology in Plain English

The researchers start by writing structured specifications for 28 public training benchmarks and 18 unseen benchmarks, grouping them into five requirement domains (rule discovery, constrained problem solving, perception and control, multi-agent interaction, long-horizon environments). A base Harness is run over sampled training instances to collect trajectories; only training records are ever used for tree building, world creation, module learning, or router training, and the train/unseen split is frozen before any capability induction begins.

Evolver reads those trajectories and proposes capability categories, which a judge consolidates and reviewers independently check for definitions, evidence, assignment and coverage. Each training task gets a fixed requirement set, and requirements are sorted by a deterministic order fixed in advance to form paths; a trie over those paths merges common prefixes into a hierarchical capability tree. The ordering is explicitly not a claim of causal dependency.

Creator then builds training worlds. For heterogeneous tasks sitting at the same node, it designs simplified levels from the node's requirements and representative public specifications — deliberately excluding target-task layouts, private proofs, and solution traces — while preserving the core decision process. A host-executed verifier pool checks node binding, execution, state transitions, rendering and completion, and a frozen screening Harness admits only worlds falling inside a predefined intermediate-performance band; otherwise Creator adjusts difficulty. Original benchmarks are what measure transfer.

Explorer evolves along the same tree. At each node it inherits parent modules selectively and refines local ones, with the host validating evidence, ownership, interfaces, lineage and typed input–output assertions. Inherited registries are composed by right-biased union so that local modules override ancestor modules, and an updated parent is kept only when it strictly beats the previous version on host-verified task-completion counts under identical validation tasks, trial counts and budgets. Updates accumulate in a fixed topology across iterations; sibling branches evolve independently.

Consolidation. At the end of each iteration, Creator merges validated module proposals — merging partial overlaps, keeping the earlier-learned module among near-duplicates — into the next Base Harness, which seeds the next round.

Routing treats Harness selection as a context–action–feedback loop: the agent sees the public goal, initial observation, metadata and images, plus candidate descriptions, retrieved experience (cosine similarity ≥ 0.5, at most ten records, 20,000-record FIFO memory) and observed performance–cost summaries, commits a choice with a rationale, and only after terminal receipts are verified does the outcome update memory. Unseen capability labels are hash-locked away from routing until the prediction ledger is final.

Evaluation. All variants share Base Harness H₀ (Prime Agent) and a frozen Claude Opus 5 backbone. Because a Harness changes how skills are learned, the authors run five preparation trials per benchmark–Harness pair, summarize those trajectories into skills, and freeze them for all subsequent Train and Unseen evaluation.

Why This Matters

Impact on research. The paper reframes self-improvement away from "one Harness per task" toward "one module tree over shared capabilities," and provides concrete evidence that simulated, capability-targeted environments can outperform training on the original benchmarks themselves at lower inference cost. The capability–module correlation analysis and the add/remove interventions give a template for empirically justifying scaffold design choices rather than choosing them by intuition, and the negative correlations (−0.361, −0.313) show that capabilities can trade off against protocols, not just reinforce them.

Real-world applications:

• Robotics deployment: Capability-evolved Harnesses plus five-episode skill preparation raised real-robot success from 0% to 60% on SOON (desk approach within 30 cm) and from 0% to 60% on CaP-X (locating a cube under inverted cups), without touching model weights.
• GUI and computer-use agents: The vision, memory, planning and orchestration protocols evolved here map directly onto desktop automation, form filling, and software QA agents that must track state over long horizons.
• Cost-aware model serving: Routing among specialized Harnesses achieved near-oracle accuracy at lower token cost than random selection, which matters for any team paying per-token for long agent episodes.
• Simulation-first training pipelines: The Real2Sim2Real recipe shows how to generate compact, verifiable training environments from public specifications alone, useful wherever collecting real interaction data is expensive or unsafe.

Industry relevance. Because the backbone is frozen, all the gains come from the surrounding orchestration layer — precisely the part that product teams actually control. The router's use of only public task information and verified past experience makes it a deployable component rather than an oracle-only construct, and the iteration-level cost reporting (training tokens per iteration, with world reuse amortizing Creator's overhead) speaks to the economics of running such a loop in practice.

Future Directions

• Why do some benchmarks regress? Eleusis declined from 82.2 to 81.1 and Mastermind stayed flat at 88.9 across Iterations 1–4, and Crafter and DiG-bench remain below both Codex and DeepSeek references; the paper does not report a diagnosis or a mechanism for detecting and correcting such stagnation.
• Isolating module evolution from skill preparation. The authors explicitly note that the embodied comparisons evaluate complete Harness–skill configurations and do not separate the effects of module evolution from those of skill preparation; a controlled decomposition is an open question.
• Beyond a fixed tree topology. Evolution accumulates within a fixed topology, with capability categories and the ordering fixed before construction. Whether the tree should grow, split, or re-order as new capability demands appear — and how that interacts with the hash-locked unseen evaluation protocol — is not addressed.
• Richer routing and memory policies. The router uses a single similarity threshold, a ten-record retrieval cap, and a 20,000-record FIFO memory with public information only; whether learned routing, uncertainty-aware fallbacks (invalid selections currently receive no fallback and technical failures score zero), or cross-iteration memory consolidation would close the remaining gap to Oracle (71.27/60.96 versus 73.16/63.61) is left open.

Target Audience

Researchers and engineers working on LLM/MLLM agent scaffolds, agent–environment co-evolution, and automated agent design; practitioners building long-horizon multimodal agents (GUI, embodied, game-playing) who need principled ways to specialize and route orchestration logic around a frozen model; and evaluation researchers interested in capability-based rather than benchmark-based task organization. Readers should be comfortable with formal notation and with reading benchmark tables, as the argument rests heavily on comparative numbers across five tables and eight figures.

Authors’ abstract

Multimodal intelligence can be viewed as human-guided co-evolution between worlds and agents through interaction. However, scaling this process faces two bottlenecks: constructing heterogeneous, verifiable environments is costly, and manual Harness iteration cannot keep pace with growing task diversity and complexity. A key insight is that tasks with different modalities and surface forms can share the same underlying capability requirements. We therefore introduce The Trinity, a recursive self-improvement framework coupling world generation with Harness evolution through three roles: Creator, Explorer, and Evolver. Drawing on verified execution trajectories, Evolver shifts the basis of task organization from surface categories to shared requirements, reorganizing the original task tree into a hierarchical capability tree. We introduce Capability-Driven Real2Sim2Real: for heterogeneous tasks at the same node, Creator abstracts their common requirements into simplified, verifiable simulated worlds for capability learning, while evaluation on original tasks measures transfer. Along this capability tree, Explorer progressively evolves reusable capability modules through selective inheritance, forming a Harness tree whose validated updates accumulate across iterations within a fixed topology. The resulting Harnesses specialize in different capabilities, motivating a Capability-Aware Harness Router that uses public task information to select compatible module configurations from the tree. Our evolved Harnesses improve mean performance by 34 points across 28 training benchmarks and 26 points across 18 unseen benchmarks, including two real-world embodied tasks.