RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent

AI summary

Overview

• Research area: Large language model agents and recursive self-improvement (RSI) — specifically, how an agent system decides which part of itself to improve, and how the mechanism making that decision can itself get better over time.
• Technical level: Advanced. The paper assumes familiarity with LLM agents, harness/prompt-level engineering, supervised fine-tuning (SFT), reinforcement learning from group-relative rewards (GRPO), and LoRA.
• Scope (one sentence): RSIFlow is a framework in which a meta-agent diagnoses a task-agent's current bottleneck, picks one of three improvement workflows (Model, Harness, or Artifacts), and distills the outcome of each round into reusable skills and principles that rewrite its own harness.

What This Paper Is About

Agent self-improvement systems can already edit a model's weights, its execution harness, or the artifacts it produces, and "meta-level" methods make the improvement procedure itself editable. What they lack is a way to decide which improvement to run given the agent's shifting weaknesses, plus a way to get better at making that decision. RSIFlow addresses both at once: a meta-agent reads task trajectories and environment feedback, routes the next improvement round to the component that is currently the bottleneck, and then converts what happened into reusable knowledge that updates its own orchestration harness.

Key Contributions

1. Formulates dynamic workflow adaptation for RSI. The paper names the coupling between evolving performance bottlenecks and the continual refinement of workflow orchestration as a distinct challenge, arguing that broad editability alone does not establish that a workflow targets the agent's actual current weakness.
2. Proposes RSIFlow, a framework that instantiates three component-specific workflows (Harness, Model, Artifacts), has a meta-agent select among them from execution feedback, and closes a loop where workflow outcomes revise the meta-agent's own harness (execution procedures, skill retrieval, planning).
3. Introduces multi-level skill formation with evidence-driven maintenance. Outcomes are distilled into component-specific skills and cross-component principles, each stored as a structured record (applicability conditions, update scope, conditional guidance, preservation requirements, expected behavioral response, supporting and contradicting evidence), and maintained through four operations: Add, Supplement, Revise, and Merge.
4. Demonstrates empirical gains across six benchmarks and two backbone models, reporting state-of-the-art average performance on all benchmarks, leading in 11 of the 12 benchmark–backbone settings, plus ablations isolating adaptive orchestration and meta-harness evolution, and a budget–performance Pareto analysis.

Main Findings

• Overall results: RSIFlow reaches overall scores of 51.21% with Qwen3-4B and 59.86% with Qwen3-8B, surpassing the strongest baseline by 6.06% and 3.86% respectively, and averages +3.11% over the strongest per-metric baselines.
• Coverage of wins: It leads in 11 of 12 benchmark–backbone settings. The single setting it does not win is ACEBench with Qwen3-8B, where HarnessForge reports 59.07 against RSIFlow's 52.62 in Table 1.
• Largest gain on code: On HumanEval+, pass@1 improves by 14.02% with Qwen3-4B; on MBPP+, pass@1 improves by an average of 3.91% across backbones.
• Tool use: Highest BFCL-v3 scores with both backbones, and the best ACEBench result with Qwen3-4B (38.00).
• Retrieval-intensive QA: F1 improves by an average of 5.20% on 2Wiki across backbones and by 6.38% on HotpotQA with Qwen3-8B; the Qwen3-8B HotpotQA score is 62.67.
• Adaptive routing beats fixed routines: Against the Vanilla starting point (Code 48.25, Tool Use 29.70, SearchQA 34.52), RSIFlow reaches 67.29, 33.62 and 52.72 — gains of 19.04%, 3.92% and 18.20%, a three-domain average increase of 13.72%. Adaptive orchestration outperforms the fixed-component variants by 4.13%–8.81% on average and exceeds the average of all four ablated variants by 6.17%.
• Meta-evolution matters: Disabling meta-agent evolution reduces average performance by 5.74%. In the domain-level Meta-version comparison, Meta0 reaches 44.62% on SearchQA through harness updates (a 10.10% gain), while Meta4, also selecting the harness, beats Meta0 by 20.56% on Code and 9.51% on SearchQA; Meta3 achieves the top Tool Use score of 31.83% through artifact updates.
• Meta-backbone sensitivity: Over five rounds on HumanEval+, gains are 11.48% (Claude-Sonnet-5), 5.09% (DeepSeek-V4.1-Flash) and 24.41% (GPT-5.6-Sol); GPT-5.6-Sol reaches the highest final performance of 68.29% but consumes more API tokens. From Round 3 to Round 5 it improves 13.08%, versus 6.15% and 4.17% for the other two.
• Budget efficiency: RSIFlow generally lies on or near the empirical Pareto frontier of the rollout-budget analysis and achieves the highest observed score on each of the six benchmarks.
• Case study trajectory: A five-round Harness–Harness–Model–Model–Artifacts sequence moves from 32.23% (output-contract check) to 38.04% (receipt-grounded verification) to 45.46% (LoRA fine-tuning with 3,798 replay calls) to 47.11% (5,397 calls) to 51.21%, an overall improvement of 18.98%.

Methodology in Plain English

The system has two agents. The Task-Agent is the worker: it has model weights, an external harness (broken into Action, Skill and Planning parts), and task-level artifacts such as generated code or answers. The Meta-Agent is the supervisor: its base model parameters stay frozen, and it only changes through edits to its own harness.

Each round, the Meta-Agent looks at how the Task-Agent performed — its trajectories and the feedback from the environment — and diagnoses where the bottleneck is. It then picks exactly one of three intervention routes, leaving the other two untouched:

• Model update: turn successful trajectories into training data and run SFT (an RL variant is also evaluated) to produce new weights.
• Harness update: choose which harness components to touch, then run a localize–refine–generate–validate procedure to rewrite prompts, memory retrieval, planning rules, tool orchestration or verification, subject to interface and permission constraints.
• Artifacts refinement: keep model and harness frozen, generate up to a bounded number of revised outputs guided by diagnostic feedback, and accept the first valid candidate with a positive score gain.

All candidates share the same parent and the same evaluation batch, so comparisons are fair; if no candidate yields a positive gain, the Task-Agent simply stays as it was. After the round, the Meta-Agent compares parent and candidate trajectories at shared subgoals, tool events and error points, and distills what worked and what failed into structured skill records and general principles. Those records enter a growing library through Add / Supplement / Revise / Merge operations that keep provenance and evidence, and the library is wired back into the Meta-Agent's Action interfaces, retrieval calls and planning — which is the recursive step.

The experimental protocol trains on 3,000 tasks split into five disjoint batches of 600 (200 code, 200 tool-use, 200 multi-hop questions per round), runs R = 5 evolution rounds with at most two attempts per round, and evaluates on a fixed 744-task reporting set (342 code, 202 tool use, 200 QA). Reporting tasks are excluded from fine-tuning, and no reporting scores or item-level feedback are used for workflow selection, acceptance, skill maintenance or harness revision.

Why This Matters

Impact on research. The paper reframes self-improvement as a routing problem rather than a pure capability problem: having more editable surfaces is not the same as knowing which one to edit. It also supplies a concrete mechanism (structured, evidence-backed skill records plus harness rewriting) for making the improver itself improvable without touching its weights, and it isolates both mechanisms with ablations.

Real-world applications:

• Code-generation assistants that must decide whether a failure calls for better prompting scaffolding, fine-tuning, or simply repairing the produced code.
• Enterprise tool-calling and API agents, where failures split between wrong function selection, wrong arguments, and wrong multi-turn state handling.
• Retrieval and multi-hop question answering over document collections, where the bottleneck may be retrieval strategy, reasoning, or the answer artifacts themselves.
• Cost-sensitive deployments: the paper explicitly frames Qwen3-4B and Qwen3-8B backbones as relevant to resource-constrained agent deployments, where targeted updates beat brute-force exploration.

Industry relevance. The rollout-budget Pareto analysis addresses the practical question teams actually face — how much compute to spend per improvement — and the finding that a stronger meta-agent yields larger cumulative gains gives a concrete argument for where to spend model budget.

Future Directions

• Scaling the evaluation: the authors state that whether the benefits hold with larger backbone models and at cluster scale remains unexplored, and flag this as future work.
• Cheaper, more stable meta-evolution: the process adds computational overhead, a lower-cost Meta-Agent may compromise gains, and the stability of experience-driven Meta-Harness updates is described as not yet fully investigated.
• Counterfactual diagnosis of the meta-agent: proposed as a way to identify meta-agent deficiencies and apply targeted local modifications rather than broad rewrites.
• Meta-agent parameter optimization: the meta-agent's parameters are frozen throughout; the authors list optimizing them as a route to better evolution efficiency and stability.

Target Audience

Researchers and advanced engineers working on LLM agent architectures, agent self-improvement and recursive self-improvement, and automatic prompt/harness optimization. It is most useful to readers already comfortable with agent execution loops and post-training methods; practitioners running cost-constrained agent deployments on small-to-mid-size backbones will find the budget analysis and the model/harness/artifact routing taxonomy directly actionable. Beginners will need background in agent harnesses and fine-tuning before the methodology sections are readable.

Authors’ abstract

Large language model (LLM) agents have demonstrated strong capabilities in complex tasks, including code generation, tool use, and multi-hop question answering. Existing self-improvement methods enhance agent performance through model optimization, harness evolution, and executable artifact refinement, while recursive self-improvement (RSI) further extends optimization to the improvement mechanisms themselves. However, adapting self-improvement workflows to evolving performance bottlenecks while improving workflow orchestration through execution feedback remains a key challenge. We propose \textbf{RSIFlow}, a \textbf{R}ecursive \textbf{S}elf-\textbf{I}mprovement framework that integrates adaptive work\textbf{flow} orchestration with meta-agent capability evolution. RSIFlow diagnoses performance bottlenecks from task-agent trajectories and environmental feedback to dynamically select component-specific improvement workflows. The meta-agent then distills workflow outcomes into component-specific skills and cross-component principles, evolving its own harness to improve subsequent workflow orchestration. Experiments across six benchmarks and two backbone models show that RSIFlow achieves the SOTA average performance on all benchmarks. Ablation studies further demonstrate the contributions of adaptive workflow orchestration and meta-harness evolution. Code is available at https://github.com/1HUAN1/RSI-Flow.