Recursa: Toward Recursively Self-Improving Foundation Model Systems

AI summary

Overview

Research area: AI agent systems, domain adaptation, and recursive self-improvement of foundation model systems (applied to knowledge-intensive legal work).

Technical level: Advanced. The paper assumes familiarity with agent harnesses, tool use, context management, and multi-agent orchestration, though the core ideas are explained in accessible terms.

One-sentence scope: Recursa is a foundation model system that adapts to a new, data-poor domain by bootstrapping a text-based knowledge repository from domain materials and then refining that repository through recursive, multi-agent self-improvement using only its own model — evaluated on Harvey LAB's Firm Knowledge benchmark.

What This Paper Is About

Foundation models inevitably meet new domains whose knowledge was never in their training, and those domains rarely come with large expert-labeled datasets or reliable reward signals for further training. The authors argue that adaptation must target the whole system — model, instructions, skills, memory, and harness — not just model weights, and that it should happen without a human expert or a stronger teacher model. Recursa addresses this by having the system learn from its own task experience: it first builds domain knowledge from source materials, then repeatedly diagnoses its own failures and edits its own instructions, skills, and memory.

Key Contributions

1. Formulates "foundation model system learning." The paper defines the problem of improving a system comprising a model, instructions, skills, memory, and a harness through domain materials and task experience, and separates it into two gaps: the knowledge gap (missing facts, terminology, or knowledge of where materials live) and the behavior gap (having the information but applying it incorrectly).

2. Proposes Recursa, a two-phase recursive self-improvement system. Phase one bootstraps a knowledge repository from the domain corpus plus self-generated practice tasks; phase two refines that repository from training tasks, execution traces, and feedback — with the same model acting as both solver and learner.

3. Introduces a harness built for long-horizon work. The harness gives the model a persistent Python read-eval-print loop (REPL) for composing tools in code, an append-only event log with programmatic retrieval for context management, and multi-agent orchestration for dispatching subagents in parallel.

4. Demonstrates gains without a stronger teacher. On Harvey LAB's Firm Knowledge benchmark, Recursa raises all-pass rate on held-out test tasks from 30.0% to 55.0% with DeepSeek-V4.1-Flash and from 35.0% to 46.0% with Qwen3.8-Flash, while the harness alone already outperforms published reference configurations.

Main Findings

• The harness alone is a strong baseline. Using Qwen3.8-27B, the persistent-REPL harness reaches 35.2% all-pass with original rubrics and 37.2% with revised rubrics, versus 21.6% for the reference base agent without study and 31.0% for the reference agent after high study effort. The paper notes that evaluation and adaptation procedures differ across systems.

• Bootstrapping mainly buys efficiency, not accuracy. On all 250 tasks with DeepSeek-V4.1-Flash, bootstrapping produced 14 skills (a navigation skill relating each of the 266 matters to notes and source documents, plus 13 skills distilled from practice). It reduced interaction turns and input tokens by roughly a fifth to a quarter and raised all-pass rate slightly, while the mean criteria pass rate remained unchanged.

• Multi-learner self-improvement drives the largest gains. With DeepSeek-V4.1-Flash, the merged components K(1) reached 51.3% training and 48.0% test all-pass; the later K(2) reached 48.7% training and 55.0% test, 25 percentage points above the agent without learning. With Qwen3.8-Flash, K(1) raised training all-pass from 37.3% to 47.3% and test from 34.0% to 46.0%, 11 percentage points above no learning.

• Splitting tasks across parallel learners matters. A single leaf learner without splitting reached 34.7% training and 43.0% test all-pass with DeepSeek-V4.1-Flash (K(4)), below the multi-learner 48.7% and 55.0%. With Qwen3.8-Flash, the single learner's K(5) fell below the bootstrap baseline in training all-pass rate (30.0% versus 37.3%) and reached 37.0% on test versus 46.0% for multi-learner learning.

• Gains concentrate in all-pass rate rather than mean criteria pass rate. The authors interpret this as learning converting partially correct answers into complete successes, with the gains carrying over to test tasks that never informed learning.

• Learning changes search behavior and inclusion judgments. In one test task, Qwen3.8-Flash without learning searched only for the literal term "no-shop" and selected an agreement signed May 20, 2025; after learning it searched "no-shop," "Exclusivity," and "No Solicitation" and identified a qualifying transaction signed June 16, 2025. In another, DeepSeek-V4.1-Flash went from returning eight matters to the seven qualifying matters after learning to establish scope from engagement records; the authors report Qwen3.8-Flash makes the same correction after learning.

• The benchmark itself has flawed rubric criteria. The authors found 46 criteria in 30 tasks that conflict with the task statement or source documents, and revised them while leaving task statements, criterion IDs, and the other 220 tasks unchanged. The revisions raised most scores by up to four points and lowered one by one point.

Methodology in Plain English

The system stores everything it learns as text: instructions (system prompt, guardrails), skills (reusable procedural packages loaded on demand, following the Agent Skills format), and memory (reflections on past task executions and learning experiences). Two agent roles operate on this repository: solvers, which tackle tasks and produce answers plus execution traces, and learners, which call solvers, inspect their traces and feedback, and propose revisions.

Phase one, bootstrapping, has parallel agents read through the domain corpus, build either navigation structures (for large, heterogeneous collections) or compact exploration notes (for narrower domains), and then generate practice tasks from the materials. Learners assess those attempts and write initial skills and memory.

Phase two, recursive self-improvement, runs a four-step loop at each "leaf" learner: solve tasks in parallel, diagnose failures while separating genuine errors from noisy feedback, edit instructions and skills, then validate by re-solving affected tasks plus a sample of already-passing ones, keeping an edit only if it fixes failures without regressions. Non-leaf learners split their task set into subsets and dispatch child learners in parallel — splitting by topic, failure mode, target skill, or tool — then merge the children's revisions (removing duplicates, reconciling conflicts, generalizing recurring lessons) and reflect on the children's learning traces to choose the next round's subsets. Two efficiency concerns guide this: reducing "learning waste" (stalls and rework) and making better use of the "learning signal" (solver feedback plus learner-side validation outcomes).

The harness keeps intermediate results in Python variables rather than the model's context, truncates long outputs to disk, stores full history in an append-only event log that agents query in code, and lets agents dispatch subagents into isolated sandboxes.

Why This Matters

The paper argues that domain adaptation should target the entire system around a model, not just weights — which matters because knowledge-intensive domains like law and finance supply plenty of documents but little expert-created training data or verifiable reward. It also offers evidence that a system can improve itself without a stronger teacher model, and its evaluation uses long-form answers judged on supporting evidence rather than short exact answers or numerical metrics that permit repeated self-testing.

Real-world applications:

• Law firm knowledge work: searching a firm's matter files, locating relevant source documents, and producing long-form answers that satisfy task-specific rubrics.
• Financial analysis and compliance: the paper names banking and finance terms, credit precedents, and capital-markets offering documents among the topic subsets learners split by, and lists finance as a future domain.
• Engineering and codebase onboarding: exploration notes are explicitly designed for narrower, well-structured domains such as a single software project.
• M&A diligence and large-corpus surveying: the paper cites prior work applying recursive language models to M&A diligence, and its multi-agent orchestration targets workloads like surveying a large corpus or diagnosing hundreds of failure cases.

Industry relevance: the paper is authored entirely at Alibaba Research and frames system-level learning as the practical route to adapting deployed agents to new enterprise domains without retraining foundation models.

Future Directions

• Extending recursive self-improvement to the foundation models and the harness. The paper states this is ongoing work; in the current setup only the knowledge repository is learned, while the harness and models stay fixed, and the authors note that the accumulated repository becomes a source of training data for model learning.

• Reducing learning waste. The paper defines stalls and rework as open efficiency problems, noting that excessive splitting creates merge conflicts and duplicate investigations across children, and that learners currently revise their own splitting, solving, and editing behavior by analyzing their own traces.

• Handling noisy learning signals. Solver feedback is described as often noisy, and learners refine diagnoses iteratively across rounds; how robust this remains under unreliable feedback is left open.

• Applying Recursa to other domains. The authors name finance, engineering, and the sciences as targets, and note that experiments so far cover a limited set of domains.

• Safety and resource questions. The limitations section flags dependence on corpus quality and availability, substantial compute requirements, propagation of incorrect or biased data through the learning loop, uninvestigated impact of underlying model limitations, and privacy/security risks from autonomous exploration and retained memory.

Target Audience

AI researchers and engineers building agentic systems for knowledge-intensive enterprise domains, particularly those working on agent harnesses, context management, multi-agent orchestration, or self-improving systems. It is also relevant to practitioners in legal tech and financial services who need agents to adapt to proprietary corpora without large labeled datasets, and to evaluation researchers interested in rubric-based, long-form benchmarks. Readers should be comfortable with agent loops, tool use, and the distinction between in-context learning and weight updates.

Authors’ abstract

Foundation models in changing environments can encounter unfamiliar domain knowledge that their original training did not cover. However, new domains rarely provide large amounts of expert-created training data or reliable supervisory signals for further training. We introduce Recursa, a foundation model system that adapts to new domains by learning from its own experience without human experts or a stronger teacher model. Recursa uses a knowledge repository to store what it learns: domain-specific skills and instructions on when to apply them, along with memories of past task experiences and reflections on what to improve. Recursa bootstraps the repository from domain materials and then refines it through recursive self-improvement. It uses parallel learners to execute tasks and synthesize local improvements, then validates and merges the improvements into the knowledge repository. On Harvey LAB's Firm Knowledge benchmark, Recursa raises the all-pass rate on held-out test tasks from 30.0% to 55.0% with DeepSeek-V4.1-Flash and from 35.0% to 46.0% with Qwen3.8-Flash.