MetaShare: Failure-Aware Collective Self-Improvement for Open-Ended Agents

AI summary

Overview

Research area: Multi-agent systems and open-ended self-improving agents — specifically, how populations of agents should share what they learn across lineages.

Technical level: Advanced. The paper assumes familiarity with LLM-based evolutionary search (DGM, DGM-Hyperagents, Group-Evolving Agents) and with prompt/tool/memory scaffold editing.

Scope: A memory-and-retrieval framework called MetaShare that stores every evaluated self-modification as a signed, mechanism-level "lesson" and uses both successes and failures to steer the next generation of agents, evaluated across five domains.

What This Paper Is About

Open-ended self-improving agents solve a task while a "meta-agent" rewrites the surrounding scaffold — prompts, tools, memory, control flow. Prior systems carry forward an agent and a scalar fitness score, so a modification that made performance worse leaves only implicit history, and the next meta-agent has no compact signal telling it which mechanism to avoid. MetaShare instead makes each evaluated modification a signed record — what mechanism was tried, whether it helped or hurt, and by how much — so later agents can reuse proven mechanisms and skip known dead ends. The paper asks whether this changes the efficiency, not just the final quality, of open-ended search.

Key Contributions

1. A signed, mechanism-level unit of shared experience. The unit of cross-lineage sharing becomes a record of the mechanism attempted with its measured performance delta, plus a dedicated "avoid" channel for failures — replacing the agent-plus-scalar-fitness unit used by DGM-Hyperagents (DGM-H) and the agent/trajectory/patch exchange used by Group-Evolving Agents (GEA).
2. Task-blind search auditing. An auditor receives a scrubbed digest of strategy types and signed deltas — with task identifiers, benchmark names, and domain-revealing patterns removed — and injects absent or underrepresented mechanism categories as "Must-Try" directives without ever seeing task labels, objectives, or absolute scores.
3. Failure semantics isolated experimentally. A length-matched shuffled-failure control separates the value of a failure record's content from the value of the extra prompt tokens it occupies.
4. Cross-domain evidence. One failure-aware mechanism is tested across five heterogeneous domains, whereas earlier cross-lineage sharing was demonstrated mainly on coding.

Main Findings

• Final quality matches or exceeds DGM-H on all five domains (medians over five runs): paper review 0.71 → 0.79; Polyglot coding 0.28 → 0.34 on the full 225 tasks with the backbone held on o3-mini; robotics reward design 0.37 → 0.40; the three IMO-grading settings 0.61/0.64/0.60 → 0.64/0.70/0.67; audit risk 0.45 → 0.55.
• Convergence is 1.7–2.9× faster measured as the generation at which each arm's selected best checkpoint appears — for example 100 → 36 generations on paper review and 44 → 26 on audit risk. Because the paper-review baseline's best coincides with its budget, that particular ratio is described as a lower bound.
• NMAE is reported as 0.175 vs. 0.157 (IMO-full) and 0.357 vs. 0.312 (audit risk). The excerpt does not label which arm each value belongs to, so the ordering is left as printed.
• Competitive with GEA on its own Polyglot protocol. GEA reports 0.88 on the 60-task small+medium set from a single run with Claude Sonnet-4.5 as both task- and meta-agent; MetaShare reaches 0.912 [0.902, 0.922] over five runs within 10 iterations. The authors explicitly read this as competitive rather than a win, since a five-run interval and a single point estimate are not like-for-like.
• The memory structure, not extra proposals, drives the gain. On paper review, a compute-matched reference with the pool disabled at k = 3 reaches 0.72 [.72, .73] validation in 55 generations, versus the full system's 0.83 [.80, .88] in 36 generations — +0.11 validation and 19 fewer generations, with signed memory alone supplying +0.06 and the auditor +0.05.
• Failure content matters, not failure tokens. On Polyglot, replacing true failure records with length-matched shuffled failures costs 0.07 (0.45 → 0.38).
• The auditor costs 0.15 when removed (0.45 → 0.30), splitting into a search effect under a fixed backbone (0.30 → 0.34) and a model-routing effect (0.34 → 0.45).
• The auditor induces a capability the baseline never finds. After 20–30 iterations it emits a "Dynamic Model Selection" directive and the meta-agent upgrades the task-agent from GPT o3-mini to GPT 4o-mini, reproducibly across runs; DGM-H, given the identical action space, never proposes it. Runs that keep o3-mini throughout still beat the DGM-H baseline in fewer iterations.
• Token cost stays in the same order of magnitude but is not uniformly cheaper. MetaShare is more expensive on two of the three domains accounted for: paper review 83M → 52M (0.6×), Polyglot 135M → 163M (1.2×), IMO-grading 88M → 213M (2.4×).

Methodology in Plain English

Recording. Whenever a generation produces a patch that applies cleanly and gets a valid score, the system computes the child's score minus the parent's score and writes a record: the modification, its mechanism type, a short description, the signed delta, both raw scores, the domain, the source-parent identifier, and the generation. Records are written only after evaluation, so they are observed outcomes rather than self-reports. The mechanism type is drawn from a fixed six-item vocabulary: prompt engineering, tool creation, memory, evaluation logic, self-modification, and structural refactoring. The pool is capacity-bounded at 100 records; on overflow the records with the largest absolute delta survive, so both the strongest successes and the most informative failures are kept.

Splitting and retrieving. The pool splits by the sign of the delta into improvements (δ > 0) and non-improvements (δ ≤ 0). Before each generation, records from the same source parent are filtered out to encourage cross-branch transfer. Retrieval is deliberately asymmetric: up to K_s = 5 positive records are pulled with a diversity-aware top-K that takes at most one representative per mechanism type before filling the rest by score, and up to K_f = 5 of the most negative failures are appended in a separate section labeled as reducing performance. The formatted context is roughly 20k tokens.

De-duplication. Positive strategies are embedded with text-embedding-ada-002 and compared by cosine similarity; near-duplicates above a threshold τ (0.92 for paper review and Polyglot, 0.85 for IMO, robotics, and audit risk) are collapsed in favor of the highest-delta member. This happens at retrieval time so the pool itself keeps related attempts for analysis.

Auditing. Every C = 10 generations (and only once at least five records exist), a task-blind auditor sees a scrubbed digest of types, signed deltas, and added lines, and names underrepresented mechanism classes. Its output is used only after passing a leak check against blocked tokens and patterns; a failed check is retried once, then rejected, with the prior valid directive retained. Valid directives are prepended as a "Must-Try" instruction.

Proposing and selecting. K = 3 meta-agents propose patches in parallel in independent containers and repo copies, at per-domain temperatures that always include a greedy 0.0 slot plus moderate and exploratory slots (e.g., [0.0, 0.35, 0.7] for paper review, [0.0, 0.45, 0.75] for Polyglot). Only one proposal is evaluated, so extra proposals raise self-modification cost but not domain-evaluation cost. Selection is structural rather than predictive: a patch must apply cleanly, stay in scope, compile as Python, and fall within 5 to 300 lines; ties are broken by a generation-seeded random choice. No learned judge ranks candidates. A staged screen (10 → 100 tasks for paper review and IMO grading, 3 → 6 repetitions for robotics, 10 → +50 for Polyglot) gates full evaluation behind a threshold γ, which is 0 everywhere except Polyglot at 0.4.

Evaluation protocol. Four domains are inherited from the DGM-H suite; the fifth, financial audit-risk assessment, is newly curated from the 10-K Financial Notes of 15 real audit clients across 15 Inherent Risk Factors, giving 225 labelled (client, IRF) pairs split 8 train / 2 validation / 5 test, with the test fold covering 75 decisions within five filings and three labels (High, Medium, None). Every condition is run five times; medians and 95% bootstrap confidence intervals are reported. On Polyglot and audit risk the baseline is the authors' own reimplementation (MetaShare with the pool disabled and a single meta-agent, same gate); on paper review, robotics, and IMO the DGM-H numbers are taken from the published work rather than rerun.

Why This Matters

Research impact. The paper reframes what self-improving agent populations should inherit — a signed, distilled, non-redundant lesson rather than a trace or a fitness number — and backs the reframing with a compute-matched ablation that separates memory structure from raw compute. It also shows that a task-blind auditor can unlock a strategy (dynamic model routing) that a task-aware baseline with the identical action space never discovers, which is a concrete argument that some useful structure does not emerge unaided.

Real-world applications:

• Financial audit-risk assessment, where the curated dataset tests document-grounded judgement over 10-K notes.
• Software engineering agents, since Polyglot spans code generation across languages and the mechanism vocabulary is coding-adjacent.
• Robotics reward design, where generated reward functions for quadruped locomotion are scored by average fitness (simulation only).
• Evaluation-heavy workflows such as math solution grading, where the task is scoring Olympiad-level solutions rather than solving them.

Industry relevance. The token accounting — 52M vs. 83M on paper review, 163M vs. 135M on Polyglot, 213M vs. 88M on IMO — gives practitioners a concrete picture of when failure-aware memory pays for itself in search time versus raw spend. The paper's own responsible-use statement stresses that the method is a research artifact for sandboxed, bounded-action-space settings with an accountable human owner, and that compressing the search from 1.7–2.9× fewer generations compresses the oversight window, so inspection should be budgeted per unit of change rather than per iteration.

Future Directions

• Evolving the scaffolding itself. The gate, retrieval rules, and auditor cadence are fixed rather than evolved, and the size band (5 to 300 lines) is not ablated; letting a hyperagent evolve this machinery is named as future work.
• Cross-branch pollination and adaptive auditor cadence. Both are listed as extensions, alongside a noted tension that a failure channel scored on a single scalar can persist caution-improving mechanisms as avoid-signals — the authors suggest exempting designated mechanism classes from the negative channel or scoring them under a separate objective.
• Validating the record classifier. Mechanism type and description come from an unvalidated LLM classifier, and an unvalidated classifier can misdirect retrieval population-wide; stronger validation is an open problem.
• More robust comparisons. Only Polyglot and audit risk rerun DGM-H behind the authors' gate, results rest on five runs with often-overlapping CIs, and type-diverse retrieval and semantic de-ranking are kept on in every ablation arm rather than independently isolated — leaving efficiency as the robust signal and quality as the softer one.

Target Audience

Researchers and advanced engineers working on self-improving LLM agents, evolutionary or population-based agent search, and agent memory architectures. It is also relevant to practitioners in regulated, evaluation-heavy domains — audit, code generation, reward design, grading — who need to justify the compute and oversight cost of autonomous self-modification. Readers without background in scaffold-search systems such as DGM, DGM-Hyperagents, or GEA will need to consult those papers first, since the contribution is defined relative to what those systems do and do not carry forward.

Authors’ abstract

Humans often learn fastest not by repeating success, but by remembering which failures not to repeat. Open-ended self-improving agents can share experience across lineages, as in GEA, or edit their own improvement mechanisms, as in DGM-Hyperagents, yet failed improvement attempts are rarely retained as reusable cross-branch knowledge, and search can collapse onto redundant strategies. We introduce MetaShare, a failure-aware framework that records successful and unsuccessful agent modifications together with signed performance deltas. Before each generation, MetaShare retrieves diverse high-performing strategies alongside high-confidence failures from other lineages, letting agents reuse effective mechanisms while avoiding known dead ends. A task-blind auditor periodically injects absent or underrepresented mechanism categories, while embedding-based de-ranking limits redundancy during retrieval. Multiple meta-agents propose modifications in parallel at heterogeneous temperatures behind a deterministic selector that gates applicability, scope, compilability, and size. Experiments across paper review, coding, math grading, robotics reward design, and financial audit-risk assessment show that MetaShare's median matches or exceeds DGM-Hyperagents' final performance and reaches its own best-performing agent in 1.7--2.9$\times$ fewer evolution iterations.