Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnesses

AI summary

Overview

Research area: Large language model (LLM) agents — specifically the external "harness" layer that coordinates memory, skill invocation, task-state tracking, tool use, and verification around a frozen model, applied to long-horizon tasks and recursive self-improvement (RSI).

Technical level: Advanced. The paper uses formal notation for state updates, patch spaces, and admission gates, and assumes familiarity with agent benchmarks and memory architectures.

Scope: The paper introduces Recuris, a recursive Experiential–Working Memory architecture in which a verified working state selects skills from an experiential memory within a task, and a fixed Meta-Agent evolves four memory components across tasks through localized, validation-gated patches.

What This Paper Is About

Long-horizon agents accumulate large interaction histories, and as those histories grow, agents lose track of unresolved goals and invoke skills that no longer match the current task state. The paper argues the missing ingredient is not more stored experience but a compact, verified task state that continuously aligns memory with what the agent needs right now.

Recuris supplies that state through a Working Memory that tracks each goal as pending, done, or blocked, and uses it to retrieve skills from an Experiential Memory at defined execution events. The same coupling turns execution into structured diagnostic evidence, letting a fixed Meta-Agent attribute failures to one of four memory components and patch only those components, subject to a validation gate on held-out tasks.

Key Contributions

1. State-grounded memory use as an RSI requirement. The paper establishes that long-horizon recursive self-improvement requires continual alignment between accumulated experience and evolving task states, rather than retrieval from the initial instruction or the full interaction history.

2. The Recuris architecture. A recursive Experiential–Working Memory design that couples persistent skills with dynamically maintained, evidence-grounded task states, enabling adaptive skill invocation throughout long-horizon execution.

3. Execution as structured diagnostic evidence. EM–WM coupling records a structured trace linking working state, invoked skills, actions, observations, proposed state updates, and checker decisions, enabling component-level failure localization and targeted evolution of Skill Memory.

4. Broad empirical validation. Evaluation on four long-horizon benchmarks and ten models, covering execution reliability, skill invocation, failure localization, evolution stability, and cross-task transfer, with gains observed across model scale.

Main Findings

• Success improves in 35 of 37 completed model–benchmark pairs. Across four long-horizon benchmarks and ten models spanning a 3B open-weight model to frontier models, Recuris raises task success in nearly every completed comparison.

• Large frontier-model gains on τ²-Bench. On τ²-Bench it adds +17.8 points to GPT-5.6 Sol and +15.6 points to Claude Opus 5, taking Opus 5 to 87.9%. On τ²-Retail specifically, GPT-5.6 Sol moves from 58.3 to 76.1 and Claude Opus 5 from 72.4 to 87.9, which is 9.7 points above the best any model in the evaluation reaches on that benchmark without Recuris.

• Strong gains on SkillFlow for the Qwen3.6 models. Qwen3.6-27B rises from 42.2 to 58.7 (+16.6) and Qwen3.6-35B from 35.3 to 48.8 (+13.5).

• The deployment model benefits most. On τ²-Retail the deployment model, doubao-seed-2-0-pro, improves by 23.3 points (58.1 to 81.4) and on SkillFlow by 16.8 points (34.6 to 51.4), both with intervals excluding zero.

• The advantage widens with interaction horizon. Gains grow to +32.2 points on the longest tasks, and common long-horizon failures fall by up to 80%. Recuris leads the base agent in all four length quartiles of τ²-Retail, by between +17.0 and +44.7 points, with no monotone decline in length.

• The long-horizon deficit is on the write path, not retrieval. Read-action recall stays within 88.0–97.9% for every variant in every quartile, while Recuris exceeds the base agent by 26.7 points of required-write recall. The base agent ends 42% of episodes requiring a write having executed none of them, against 16% for Recuris, and the median turn of the first correct write is identical across all four variants.

• Working Memory carries the level; Experiential Memory is invocation-conditional. Against the base agent on τ²-Retail, adding EM is worth +2.0 points (interval includes zero) and adding a verified working state is worth +23.9. Against a working state alone, EM adds a further +1.5 with interval [−2.4, +5.7]. A model-controlled variant that holds the same skill library and injects all of it every turn scores 18.0 points below Recuris (interval [+11.6, +24.6]).

• Failure localization is far more accurate from the structured trace. Localizing a failure to the responsible memory component is 64.8% accurate from the structured trace against 13.0% from the task outcome alone.

• Task structure determines the adaptation regime. τ²-Retail, τ²-Airline and SkillFlow share tools, policies and task families, so repairs transfer and memory can evolve across tasks. Terminal-Bench 2.1 has no such structure, and cross-task evolution on it admitted no patch in thirteen runs of the evolution loop.

• Extra context hurts. A regime that keeps the whole skill library standing in the prompt carries 3,111 more tokens at the first call than Recuris, scores 18 points lower, and costs 46% more per success.

• The gains come from evolution, not from memory as such. The memory-control layer before anything is learned is indistinguishable from zero on four target models.

• No model weights change. The base LLM, tools, Meta-Agent, localization and patching procedures, validation gate, and harness mechanisms outside the memory-control layer all remain fixed; the deployment model's gains are obtained with the base model left exactly as its provider shipped it.

Methodology in Plain English

The harness is built around a frozen instruction-tuned model. Each task is represented by a small working state listing goals, each marked pending, done, or blocked, with the evidence supporting that status. When the agent reaches a defined execution event — such as drafting a state-changing tool call, or a turn boundary — an invocation policy consults the working state and pulls matching skills out of the experiential memory. On τ² domains the trigger is a drafted state-changing tool call, and the harness returns a synthetic not-executed result so the skill reaches the model before the action is issued. On Terminal-Bench 2.1 the trigger is a first-turn boundary, supplying all skills once at the start.

After the agent acts, the memory specification proposes an update to the working state, a set of checkers evaluates that proposal against the actual environment observation, and a fixed commit rule accepts only the supported changes. Invoking a skill or attempting a tool call is never treated as completion evidence.

The four evolvable components are the experiential memory (stored skills), the working-memory specification (state schema and update proposal), the invocation policy (trigger predicate and retrieval key), and the checker set (completion predicates). Across tasks, a fixed Meta-Agent reads the structured traces of failed runs, attributes each diagnosed failure to the component where a localized repair is most likely to help, and writes one edit per implicated component. A fixed validation gate compares the candidate against the current memory on the failed task and a held-out development set containing anchor tasks the current memory already solves; the patch is admitted only if it repairs the target failure without violating the preset regression criterion. Otherwise the memory is kept unchanged.

Each benchmark's Skill Memory is evolved once, on the deployment model, using that model's own failures through the benchmark's reference agent as the only evidence. That single memory is then reused unchanged for every other model.

Why This Matters

The paper reframes recursive self-improvement as a problem of where recursion is allowed to happen. By confining evolution to an externalized memory-control layer while keeping the base model, the Meta-Agent, the gate, and the outer harness fixed, it demonstrates measurable, compounding improvement without touching model weights — a practical and interpretable alternative to unconstrained self-modification.

Real-world applications include:

• Customer-service and transaction agents. The τ²-Retail and τ²-Airline domains are dual-control, policy-constrained tool-use dialogue, directly mapping to support agents that must satisfy a user while changing database state correctly.

• Long-running terminal and DevOps work. Terminal-Bench 2.1 covers terminal work, where the paper's test-time adaptation mode with a binary hidden verifier applies.

• Lifelong skill reuse. SkillFlow's 166 tasks in 20 families model settings where the same execution flow recurs, and an evolved package can carry shared procedure.

• Reusable agent memory as an asset. Because evolution is the expensive step and the resulting memory costs nothing to reuse, one evolved memory serves many models.

Industry relevance: The approach addresses a common failure mode in deployed agents — retrieving from an ever-growing context, which the paper shows is both less accurate and 46% more expensive per success than state-grounded invocation. It also offers a controlled update pathway for agent memory with an explicit acceptance boundary, which matters where auditability and regression control are required.

Future Directions

• Extending cross-task evolution to benchmarks without shared structure. Terminal-Bench 2.1 admitted no patch across thirteen rounds of the evolution loop, so the conditions under which cross-task evolution pays off — and what should replace it when it does not — remain open.

• Resolving the τ²-Airline result. With only 50 tasks, every interval in the airline column includes zero, so its direction is reported rather than an established effect.

• Building and comparing memories evolved on different base models. The paper deliberately reports the single-source setting, with one memory evolved on a mid-sized deployment model and reused unchanged elsewhere; whether a memory evolved on a frontier model transfers better is unexamined.

• Understanding how gains scale with model strength. The value of Recuris does not follow model scale in the reported results, and the paper attributes gains to shared task structure rather than difficulty, leaving the interaction between model capability and memory evolution unresolved.

Target Audience

Researchers and engineers working on LLM agent architectures, agent memory systems, and recursive self-improvement, especially those building long-horizon tool-use or terminal agents. The paper's distinctive contribution — coupling a verified working state to skill invocation and using the resulting trace for component-scoped, validation-gated memory evolution — will be most useful to readers already comfortable with agent harness design and benchmark evaluation, and will be harder for beginners because of its formal notation and dense ablation tables.

Authors’ abstract

Recursive self-improvement (RSI) remains hard in long-horizon tasks, where growing histories obscure the task state and misalign skill invocation. We introduce Recuris, a recursive Experiential-Working Memory architecture for long-horizon agent harnesses, in which Working Memory tracks task progress and guides skill selection from Experiential Memory, grounding skill use in current needs rather than the full history. This coupling also turns execution into structured evidence that localizes failures to specific memory components. Across tasks, a fixed Meta-Agent turns that evidence into localized, validation-gated updates to Skill Memory that reshape execution and yield new evidence, forming a bounded recursive memory-evolution loop. Across four long-horizon benchmarks and ten models, Recuris improves task success in 35 of the 37 completed model-benchmark pairs, carrying frontier models to SOTA-level task success: on tau-bench it adds +17.8 points to GPT-5.6 Sol and +15.6 to Claude Opus 5, taking Opus 5 to 87.9%, and +16.6/+13.5 points on Qwen3.6-27B/35B on SkillFlow. The advantage widens as the interaction horizon grows, to +32.2 points on the longest tasks, and common long-horizon failures fall by up to 80%. These results position recursively evolving memory as a scalable foundation for RSI, enabling agents to continuously transform accumulated experience into increasingly effective long-horizon behavior. Code: https://github.com/Gen-Verse/Recuris