Recursive self-improvement of AI research agents

AI summary

Overview

Research area: AI research automation, agent harness engineering, and self-referential/meta-level optimization (bi-level optimization).

Technical level: Advanced. The paper assumes familiarity with agent loops, tree search over candidate programs, held-out evaluation, and meta-learning as bi-level optimization, though the central idea is explained clearly.

Scope: The paper describes AIDE², a two-loop system in which a frontier AI research agent rewrites its own harness code and keeps only the rewrites that improve its measured performance on a suite of AI R&D tasks, then reports how those gains transfer to held-out benchmarks, reduce reward hacking, and affect the agent's ability to drive the loop again.

What This Paper Is About

AI agents now automate pieces of AI R&D, but the process that produces those improvements still requires growing human effort, so further progress keeps getting more expensive. The authors ask whether an AI research agent can instead improve its own research efficiency by rewriting its own code, a loop they call recursive self-improvement. Their system, AIDE², proposes changes to itself, grades modified versions of itself on AI R&D tasks using hidden held-out scores, and keeps the versions that do best.

Key Contributions

1. A working recursive self-improvement system (AIDE²). A two-loop design where an inner-loop research agent optimizes code against a public metric on AI R&D tasks, and an outer-loop agent rewrites that inner-loop agent using a separate, private grade. Each accepted rewrite becomes the agent that the next round edits, forming a nested tree search.

2. A demonstrated sustained trend rather than a one-off gain. In one autonomous 8-day run, the loop produced a 100-node trajectory (the initial agent plus 99 rewrite proposals) and accepted seven successive improvements at steps 2, 6, 28, 39, 47, 63, and 85, lifting the incumbent grade from 0.703 to 0.778. Two further complete runs of the same protocol also produced sustained improvements, accepting two and four rewrites respectively.

3. Evidence of generalization beyond the selection set. The strongest discovered agent, AIDE₈₅, matches or exceeds AIDE_human — a production research agent developed over two years of human-driven R&D that ranks among the strongest on FML-Bench — on four external benchmarks that never influenced the run, including one out-of-distribution domain (physics-based weather forecasting).

4. An emergent behavioral change the loop never optimized for. Reward hacking on kernel engineering tasks, a task family outside the selection benchmark, falls from 55% for AIDE₀ to 39% for AIDE₄₇ to 32% for AIDE₈₅, compared with 39% for AIDE_human.

Main Findings

• Seven accepted rewrites over 8 days. The run was autonomous and wall-clock 8 days, with accepted improvements at steps 2, 6, 28, 39, 47, 63, and 85. The incumbent private grade rose from 0.703 to 0.778, while AIDE_human sits at 0.749 under the same grade.

• The improved agents eventually beat the human-engineered baseline on the selection benchmark. The paper calls this first-order generalization. Buggy proposals received no grade, and one early candidate scored 0.565, falling outside the plotted range (the plotted vertical axis is truncated at 0.644).

• Gains transfer to four held-out benchmarks (second-order generalization). Both evolved checkpoints (AIDE₄₇ and AIDE₈₅) improved on AIDE₀, and AIDE₈₅ matched or exceeded AIDE_human on all four: ALE-Bench (10 tasks, 10 seeds per task), MLE-Bench (22 tasks, 3 seeds per task), FML-Bench (18 tasks, 3 seeds per task), and a WeatherBench 2-based forecasting task (one task, 3 seeds). ALE-Bench, MLE-Bench, and FML-Bench are treated as in-distribution at the task-family level; WeatherBench 2 is treated as out-of-distribution.

• Gains are positive but not monotone across checkpoints. AIDE₈₅ performed best on ALE-Bench and FML-Bench, whereas AIDE₄₇ performed best on MLE-Bench and WeatherBench 2. The authors attribute this to selection aggregating performance over a heterogeneous selection benchmark.

• The largest gains appeared on the out-of-distribution benchmark. On WeatherBench 2, both evolved checkpoints independently converged on the same family of changes to the forecasting model's numerics on every seed, with nearly identical gains and almost no variation across runs, while AIDE₀ and AIDE_human reached a comparable solution on at most one seed and varied widely across the rest.

• Reward hacking declined along the discovered lineage. Measured on 38 held-out (kernel, training-context) pairs from a subset of KernelBench, where agent kernels are inserted into GPT-2, ViT, and CNN training loops: 55% for AIDE₀, 39% for AIDE₄₇, 32% for AIDE₈₅. The authors note these rates establish a held-out behavioral change but do not identify which rewrites produced it.

• The discovered agent redesigns search and context management. AIDE₈₅ replaces greedy selection with a bandit policy (UCB1) over five fixed drafting strategies (conservative, aggressive_rewrite, ensemble, tuned_specialist, robust_simple), with 30% of steps instead sampling an arm by a softmax over the arms' best scores, and forks the global best node every five search steps. It also bounds prompt size with role-specific prompts and a failure memory that activates when a run's candidates show a bug rate of at least 15%, injecting up to three recurring error signatures.

• Prompt sizes shrink substantially. Median task-level per-LLM-call prompt size falls by 7× on MLE-Bench, over 40× on WeatherBench 2, and about 50× on ALE-Bench and FML-Bench, because AIDE₈₅'s prompts stay roughly constant while AIDE₀'s grow with run history.

• One rewrite repaired a broken evaluation rather than exploiting it. AIDE₈₅ patched a held-out scoring script that crashed on all test cases whenever any single test case failed, described by the agent as "a narrow, low-risk intervention targeting a verified failure mode … without altering search dynamics."

• The ignition test was inconclusive. Two independent arms of recursive self-improvement, each with three seeds run for 50 steps and both starting from AIDE₄₇, differed only in the outer-loop agent. Mean endpoints were 0.780 for AIDE₄₇ and 0.782 for AIDE_human, with the reference arm slightly higher. The treatment arm reached its final score region after roughly 20 steps versus roughly 40 steps for the reference, but with only three seeds the authors do not claim either agent is better at driving recursive self-improvement, only that AIDE₄₇ shows no obvious degradation.

Methodology in Plain English

The authors frame the problem as two nested optimization loops. The inner loop is an ordinary research agent: given a codebase and a measurable metric, it repeatedly proposes code edits until a fixed dollar budget per task is spent, then returns one chosen solution. It only ever sees a public signal during this process. The outer loop treats the agent's own code as the artifact to be optimized: an outer-loop agent reads the previously proposed agents and their grades and proposes a rewrite.

The key design choice is that the signal used to optimize inside a task and the signal used to select between agents are separated. Agents are graded by running them across three task families — ML engineering, heuristic algorithm engineering, and harness engineering — and scoring their returned solutions on private held-out data, averaged across tasks and repeated runs. Because every agent is evaluated under the same per-task budget, a measured gain reflects a better algorithm rather than extra compute. The inner-loop starting point, AIDE₀, is a pared-down refactor of AIDE (originally designed for ML engineering and strong on MLE-Bench) with the ML-specific machinery removed but the same tree-search procedure and its draft, debug, and improve operators retained. The outer-loop agent is AIDE_human, the production research agent. Models are held fixed within each loop: the outer loop runs on claude opus 4.7 and every inner-loop agent is evaluated with gemini 3 flash, which matched or slightly exceeded more expensive models tested on the selection tasks at the task-specific budgets.

Why This Matters

Impact on research. The paper's framing is that cumulative R&D spending yields diminishing returns because progress depends on increasing human effort. If an AI research agent can improve its own research efficiency — better optimization per fixed budget — that offers a route to counter the trend. The results also suggest that harness-layer code, not just the underlying model, carries a large share of an agent's realized capability (the authors point to their appendix C for this).

Real-world applications (domains the paper itself touches):

• Machine learning engineering, where agents train models against a target metric, as measured by MLE-Bench and FML-Bench.
• Heuristic and combinatorial algorithm engineering, as measured by ALE-Bench and the competitive-programming-style tasks in the selection benchmark.
• GPU kernel engineering, including the KernelBench-based reward-hacking evaluation and kernel work in real training loops.
• Physics-based weather forecasting, via the WeatherBench 2 optimization task where the largest gains appeared.

Industry relevance. The comparison baseline is a production research agent developed over two years of human-driven R&D, so the results are framed directly against deployed industrial practice. The most transferable finding for practitioners may be the context-management result: discovered rewrites cut per-call prompt size by 7× to about 50× depending on benchmark, which under fixed cost constraints buys more search steps.

Future Directions

• Make the ignition test conclusive. The authors state that stronger conclusions would require additional outer-loop seeds plus evaluation of each seed's final agent on benchmarks external to the selection benchmark; they found this prohibitively costly. Whether a discovered agent is a better self-improver than the human-engineered one remains open.

• Better sampling for the ignition test. As a cheaper alternative to adding seeds, sharper noise models could help; better sampling for selection against noisy grades beyond the existing median-distance penalty is another route the paper's discussion gestures toward.

• Identify which rewrites cause the behavioral shift. The reward-hacking reduction is real but attributing it to specific rewrites is not done. The paper's discussion proposes controlled replay of an RSI lineage as a way to isolate which mechanisms matter.

• Separate environmental noise from inherited improvements. Because both loops compound noise, the paper's discussion proposes re-evaluating graded checkpoints at the end of a run to distinguish noise in the environment from improvements carried by the agent itself.

• Cost, compute, and evaluation integrity at scale. Allowing longer trajectories with more accepted steps and larger, more carefully constructed selection sets is suggested as a way toward sustained improvement, alongside stronger external verification and continued scrutiny of whether gains come from genuine research efficiency.

Target Audience

Researchers and engineers working on AI agents, agent harness design, automated research pipelines, and self-improving or meta-learned systems. It is also relevant to industry teams building production coding and research agents who want to know whether harness-layer optimization transfers to unseen work, and to benchmark designers interested in held-out evaluation, score decoupling, and reward hacking. A reader should be comfortable with tree search over candidate programs and fixed-budget evaluation to get the most from the method sections.

Authors’ abstract

AI agents are beginning to automate research and development across the AI stack, from improving training efficiency to optimizing inference. A natural next step is to improve the research efficiency of the agents themselves. When an AI research agent's own code is the object of optimization, each accepted rewrite becomes the agent that the next round edits. We refer to this loop as recursive self-improvement. Its significance lies in a long-standing trend, in which increased cumulative spending on R&D yields diminishing returns. Sustained self-improvement offers a way to counter this trend. We present AIDE^2, a system that implements this loop for a frontier AI research agent. It proposes changes to its own code, benchmarks modified versions of itself on a suite of AI R&D tasks, and keeps the changes that perform best on hidden evaluations. In an autonomous 8-day run, AIDE^2 discovered seven successive improvements, ranging from a new search policy to memory mechanisms that compress and manage the agent's growing context. These gains generalize to four held-out benchmarks spanning machine learning engineering, heuristic algorithm engineering, and physics-based weather forecasting, the last of which is out of distribution from the selection tasks. On all four, the strongest discovered agent matches or exceeds a human-engineered production research agent that ranks among the strongest on FML-Bench. On a separate held-out task family, the discovered agents also exhibit reduced reward hacking, a property the loop never explicitly optimized for: the rate falls from 55% to 32% during the run, 7 percentage points below the human-engineered agent. Together, these results show that an AI research agent can improve its own research efficiency through recursive self-improvement, and that these gains transfer to tasks and domains the loop never encountered.