CoEvo-RSI: When Harnesses Evolve, Which Models Should Follow?

AI summary

Overview

Research area: Machine learning — specifically self-improving LLM agents, agent harness/scaffold engineering, and model–harness co-evolution.

Technical level: Intermediate. The paper uses formal notation for its optimization objective and inheritance operators, but the core idea (aligning a model with the scaffold it runs inside) is intuitive. Readers should be comfortable with SFT, rejection sampling, model merging, and evolutionary search.

Scope (one sentence): The paper formalizes the "brain–body mismatch" problem in co-evolving model–harness pairs and proposes CoEvo-RSI, a framework that lets a meta-agent inherit both harness designs and model checkpoints from an evolutionary archive, evaluated on failure attribution, role-aware text-to-SQL, and smart-home instruction following with a Qwen3.5-9B backbone.

What This Paper Is About

An agent's performance depends on two things: its brain (the language model that reasons and picks actions) and its body (the harness — prompts, tools, memory, control flow). Recent work co-evolves both, but as either side changes, the harness can start demanding behaviors the model never learned, or the harness can block capabilities the model already has — for example, the model emits XML tool calls while the evolved harness expects JSON. The paper names this the brain–body mismatch problem and asks how to keep the two compatible across successive rounds of co-adaptation, rather than always pairing the newest model with the newest harness.

Key Contributions

1. Problem formalization. The authors define brain–body mismatch in model–harness co-evolution as the case where the behaviors required by harness H_t diverge from those learned by model θ_t, or where H_t restricts capabilities already present in θ_t. They also introduce the Turn-level Interface Mismatch Rate (TIMR), a cheap diagnostic counting the fraction of model turns with unavailable-tool calls, interface-contract violations, or malformed outputs.

2. CoEvo-RSI method. A bi-level framework built on bidirectional historical inheritance: a shared evolutionary archive A_<t stores earlier harnesses, checkpoints, development histories, and execution outcomes; a Curator organizes evidence; K independent harness editors each pick their own historical starting design; and model inheritance selects or merges archived checkpoints (via TIES-Merging) to match each offspring harness.

3. Multi-domain empirical evaluation. CoEvo-RSI is tested on multi-agent failure attribution (Who&When Hand/Algo., Who&When Pro, TraceElephant), role-aware text-to-SQL (RBAC Spider and BIRD), and smart-home instruction following (HomeBench VM and IM), all starting from Qwen3.5-9B.

4. Cost and inheritance analysis. The paper reports API expenditure, wall-clock run span, inference cost–accuracy Pareto comparisons, and a full harness × checkpoint TIMR matrix showing that interface compatibility is pair-specific rather than monotonic in generation number.

Main Findings

• Failure attribution gains over frontier models. CoEvo-RSI reaches 68.39% Who and 29.31% When on Who&When Hand, ahead of the strongest harness-enhanced system, FALAT + MiniMax-2.5 (66.30% and 25.90%). On Who&When Pro it reaches 69.43% Who, above Qwen3.8-Max (65.03%) — a +4.40 agent-accuracy gain. On TraceElephant it reaches 70.78% Who, +14.16 over DeepSeek-V4-Pro-0813 (56.62%).

• One benchmark where it does not lead. On Who&When Algo., CoEvo-RSI obtains 62.17% Who and 44.71% When, below the strongest baseline scores of 68.25% and 47.62% respectively. The authors also note that RL baselines (GRPO, GSPO) trained under the initial harness attain higher When accuracy on Who&When Pro and TraceElephant, even though CoEvo-RSI leads on Who accuracy everywhere.

• Single-sided evolution falls short. Self-Harness trails CoEvo-RSI by 11.49 Who points on Who&When Hand. On TraceElephant, both harness-evolution baselines drop below 10% When accuracy while CoEvo-RSI retains 28.92%.

• Role-aware SQL: best safety-utility balance on Spider. CoEvo-RSI achieves the best Safe-EX (80.85%), Safe-Deny (96.64%), and violation rate (1.57%), surpassing Pi + MiniMax-M3 (80.26%, 96.33%, 1.72%). Across the run, Safe-EX rises from 77.09% to 80.85% and violation rate falls from 2.00% to 1.57%, a 21.5% relative reduction.

• Weaker but competitive OOD SQL transfer. On the BIRD subset (never used for training), Safe-EX is 60.15% versus GLM-5.3's 60.90%, alongside the highest Safe-Deny (91.26%), the lowest violation rate (5.65%), and the second-best AC-F1 (90.24%). The authors caution that the safety–utility balance may partly reflect more conservative refusal.

• Smart-home control matches much larger models. On HomeBench valid multi-device instructions (VM), CoEvo-RSI gets the best success rate (75.51%) and F1 (89.23%), ahead of DeepSeek-V4-Pro-0813 (75.10% and 89.18%) — versus the base Qwen3.5-9B at 10.91% success and 40.15% F1. On invalid instructions (IM) it ties for best success rate (84.54%) with the highest F1 (91.80%).

• Ablations show both sides matter. Harness-only evolution has lower SA than full CoEvo-RSI in every round and TIMR above 1% in eight of ten rounds (full CoEvo-RSI: at most 0.22%). Model-only keeps near-zero TIMR but SA at most 35% — interface compatibility alone is not enough. Latest-only regresses at round 4; full CoEvo-RSI recovers by reusing round-4 weights with new harnesses at rounds 6 and 8, reaching 61.11% and 61.90% AA.

• Inheritance reaches backward productively. In the RBAC run, mean normalized TIMR drops from about 0.72 in rounds 1–3 to 0.13 in rounds 6–10, and in seven of ten rounds the selected checkpoint is the lowest-TIMR one available. Five rounds re-select existing weights, and every later derived checkpoint merges with the round-3 checkpoint. Round 10 reuses that checkpoint despite TIMR of 0.54–1.00 under harnesses 4–9, because it has the lowest TIMR (0.02) under the final harness and attains the run's best Safe-EX (80.85%).

• Robust to search budget, cheap to run. Varying K over {4, 6, 8, 10} keeps AA between 45.11% and 52.17%, SA between 28.80% and 37.50%, and TIMR at or below 0.12%. Three-round runs cost $2.06–$3.69 in API expenditure and finish in 17.48–20.71 hours, with evaluation taking 4.54–6.11 hours; the default K=8 gives the shortest span of 17.48 hours at $2.72.

• Favorable inference economics. On Who&When Hand, CoEvo-RSI reaches 68.39% at $0.16 per full evaluation pass, versus DeepSeek-V4.1-Flash at 60.34% and $3.55 — 8.05 points at 22× lower cost. On RBAC Spider it exceeds Pi + MiniMax-M3 by 0.59 Safe-EX points at 38× lower cost.

Methodology in Plain English

Start with a hand-written harness and a small model (Qwen3.5-9B), and keep an archive of every harness, every trained checkpoint, and every result produced so far. Each round, a meta-agent — DeepSeek-V4.1-Flash running in Codex — looks at how the current pair failed and acts as a "Curator," pulling relevant past designs out of the archive as evidence.

Then K=8 independent harness editors each write an improved harness. Crucially, each editor is free to pick any archived harness as its starting point, not just the current one. Once the offspring harnesses exist, a separate inheritance step decides which model should drive each one: it can pick a single archived checkpoint or blend several using TIES-Merging, guided by past task performance and TIMR. Shortlisted pairs are evaluated, the best is chosen by validation score, and that model generates its own trajectories under its harness. Only trajectories the verifier accepts are kept, and one epoch of full-parameter SFT (learning rate 5e-6, batch size 8, four H100 GPUs) adapts the model. A promotion step then picks the best of three candidates — the incumbent, the selected pair before adaptation, and the adapted pair — and requires a gain larger than a margin δ_prom to replace the incumbent. All outcomes, including failed ones, go back into the archive.

The setup used ten rounds with K=8, up to three sampled trajectories per training task at temperature 1 (stopping at the first verifier-accepted one), and 3,400 single-error training instances drawn from TracerTraj, CORRECT-Error, AFTraj, and self-constructed trajectories, with 200 TracerTraj instances held out for validation. All reported test benchmarks are out-of-distribution.

Why This Matters

Research impact. The paper reframes co-evolution as an alignment problem rather than two separate optimization loops, and shows empirically that inheriting from the archive — sometimes from a much older checkpoint — beats always using the latest generation. It also offers a lightweight diagnostic (TIMR) that other co-evolution systems can adopt without an expensive evaluation pipeline.

Real-world applications (as scoped by the authors):

• Lightweight failure-attribution evaluators for large-scale reinforcement-learning training runs.
• Local smart-home controllers that must execute feasible device commands and correctly reject infeasible ones.
• Enterprise website assistants operating under strict database-access permissions, where refusal and safe SQL execution both matter.
• Policy-constrained SQL generation where a user's role must authorize every referenced table and column.

Industry relevance. The headline economics are the draw: a 9B model reaching or beating frontier-model accuracy on domain-specific metrics at $0.16 versus $3.55 per evaluation pass, with full three-round co-evolution runs costing roughly $2 to $4 in API spend and finishing within a day. For teams deploying vertical agents under budget or privacy constraints, that is a practical alternative to calling a frontier model. The authors are explicit, however, that this is domain specialization, not general recursive self-improvement: held-out same-domain gains do not imply transfer to unrelated domains.

Future Directions

• Diagnosing capability mismatch, not just interface mismatch. TIMR only captures observable interface violations; the authors state that broader capability alignment requires complementary evaluation. A richer mismatch metric would be an obvious next step.

• Testing whether inheritance generalizes across domains. The paper's own framing separates "practical RSI within a domain" from "autonomous improvement across all domains." Whether an archive built on failure attribution helps on an unrelated task family is left open.

• Separating genuine capability from conservative refusal in safety tasks. The authors note that the safety–utility balance in role-aware SQL may partly reflect more conservative refusal, which raises the question of how to tell real policy compliance from simply declining to act.

• Extending beyond SFT-only adaptation. All adaptation here uses one epoch of supervised fine-tuning on rejection-sampled trajectories; whether stronger adaptation methods, or larger harness-candidate budgets than the tested K ∈ {4, 6, 8, 10}, change the mismatch dynamics is not reported.

Target Audience

Researchers and engineers working on LLM agents, agent scaffolds, and self-improving systems will get the most from this paper — particularly those building domain-specific agents on compact open models and looking for a principled way to decide which model checkpoint to pair with a newly evolved harness. It is also useful for practitioners who care about cost-per-accuracy in deployment, and for anyone studying model merging or evolutionary search over agent architectures. Readers without background in agent harnesses will need to absorb some scaffolding terminology first.

Authors’ abstract

Model--harness co-evolution has emerged as a prominent paradigm for recursive self-improving (RSI) agents, jointly evolving their execution scaffolds and intrinsic capabilities. We study the persistent yet often overlooked brain--body mismatch problem in this paradigm, where an evolving harness (body) demands behaviors its model (brain) has not learned or restricts capabilities it already possesses. We introduce CoEvo-RSI, a RSI framework that advances model--harness co-evolution through agent-driven, bidirectional historical inheritance. Drawing on a shared evolutionary archive and execution feedback, a meta-agent coordinates harness development from historical designs with model checkpoint selection or merging and subsequent adaptation under the selected harness, aligning inherited capabilities with evolving execution requirements to reduce brain--body mismatch. Using only supervised fine-tuning (SFT) on its own rejection-sampled trajectories, CoEvo-RSI makes Qwen3.5-9B competitive with larger models across diverse vertical applications, including multi-agent failure attribution, role-aware text-to-SQL, and smart-home instruction following. It surpasses DeepSeek-V4-Pro on TraceElephant (+14.16 agent accuracy) and GLM-5.3 on HomeBench VM (+4.90 success rate), underscoring the need for brain--body alignment and the potential of bidirectional historical inheritance to advance agent co-evolution.