REM: What to Store, When to Recall, What to Forget in Experience Memory for Coding Agents

AI summary

Overview

Research area: Memory and continual learning for large language model (LLM) agents, specifically repository-level coding agents that run for hundreds of think-act steps.

Technical level: Intermediate. The paper is written in accessible prose and its core ideas (what to store, when to inject, when to remove) are easy to grasp, but it assumes familiarity with LLM agent loops, retrieval/embedding pipelines, and benchmark evaluation with statistical tests.

Scope (one sentence): The paper introduces REM (Reusable Experience Memory), a framework that manages the full lifecycle of experience reuse — consolidating past coding trajectories into compact phase-structured documents, gating costlier implementation records behind an in-loop struggle signal, and evicting them after one exposure — and shows on SWE-ContextBench Lite (99 tasks) that this lifecycle governance, rather than retrieval accuracy alone, drives improved task resolution at lower cost.

What This Paper Is About

LLM agents now solve long-horizon tasks such as repository-level bug fixing through hundreds of think-act steps, producing rich traces of exploration, dead ends, and eventual solutions — yet that experience is rarely reused on the next run. Prior memory systems focus almost entirely on retrieval: finding the most relevant past trajectory. The problem is that retrieval cannot observe the running agent, so it cannot decide whether the agent needs an experience, at which point in the run, or for how long it should stay in context. REM reframes agent memory as a control problem over three explicit decisions — what to store, when to recall, and what to forget — made from inside the agent's think-act loop.

Key Contributions

1. A reframing from "which" to "when." The paper argues that the value of a recalled experience depends on the state of the receiving agent, making agent memory a control problem rather than a lookup problem.

2. The REM framework. A full memory lifecycle governed from inside the agent's loop: phased consolidation on the write path, a two-tier read path (a ~270-token document injected at task start plus a costlier implementation record withheld until needed), struggle-gated recall, and one-shot eviction. Each stage is a swappable policy slot, and the extraction schema — the only domain-specific artifact — is discovered automatically from example trajectories.

3. State-of-the-art retrieval plus improved resolution at lower cost. The phased document is the most accurate retrieval unit on SWE-ContextBench Lite by a 13-point margin, and REM improves resolution in every replicate while cutting cost 17%, whereas start-of-run summaries and raw trajectories do not improve resolution even when oracle-supplied.

4. Decision-level ablations and a utilization analysis. Isolating each lifecycle decision shows the stored form drives the gains, while gating and eviction keep the costly second tier from doing harm; an identifier-coverage probe links the effect to how much injected context agents actually reference.

Main Findings

• Retrieval is not the bottleneck, storage form is. REM reaches 68.7% match@1 with the automatically discovered schema (66.7% with the hand-written one), versus 55.6% for the strongest existing system (LangMem), 38.4% for OpenViking, 35.4% for a free summary, 25.3% for Mem0, 23.2% for Supermemory, and 22.2% for free trajectory search. That is a 13.1-point margin over the strongest existing system and at least 30 points over every other.

• Correct content alone does not help. Five summary and trajectory conditions resolve 22–24 of 99 tasks against 23 for no memory, including the oracle conditions that receive the labeled trajectory (23.2% vs. 23.2%). The benchmark's previously reported oracle-summary gain (34.3% vs. 26.3%) does not reproduce under this agent.

• REM improves resolution and cost together. 27.6% versus 24.9% without memory (+2.7 points, +11% relative), with per-replicate counts of 27, 29, 26 against no-memory's 23, 26, 25 in every replicate. Of 297 paired task-runs, 20 resolve under REM that fail without memory and 12 the reverse; the paired difference is not significant at 99 tasks (exact McNemar p = 0.22), so the authors rest the claim on cross-replicate consistency. Cost falls 17% (452K to 374K tokens per task), turns drop from 70.4 to 64.6, mean wall-clock time from 11.9 to 8.8 minutes, and unappliable patches from 7.0 to 4.0.

• The same oracle trajectory succeeds only when rendered as a phased document. The oracle phased document resolves 26 in a single run, while the raw oracle transcript raises cost from 452K to 768K tokens per task, shortens runs from 70 to 50 turns, and raises unappliable patches from 7 to 11 — the agent trades its own exploration for the transcript without resolving more.

• Agents read structured documents far more completely. Coverage (share of injected identifiers a run references) is 3.4% for a wrong summary, 26.7–27.2% for raw trajectories, 45.7–52.5% for prose summaries, 64.7% for the oracle phased document, and 69.1% under REM. REM's 270-token document is smaller than the 390-token oracle summary yet read 1.5× as completely, so organization rather than volume drives uptake. Among start-of-run forms, resolved runs reference more of what they were given (d = +0.20, p = 0.03; 91 resolved vs. 284 unresolved runs).

• Gating and eviction each pay off, but mainly by avoiding harm. Eager injection of the record resolves 25.3% versus 27.6% for REM and costs 423K versus 374K tokens, injecting the record in 100% versus 58% of runs. Pinning the record resolves 24.9% versus 27.6% in all three replicates despite near-identical coverage (50.3% vs. 50.5%), so eviction changes how long the record shapes later turns rather than how much is read. Never injecting the record resolves 26.9%, within 0.7 points of REM — the document carries most of the benefit.

• Memory helps a capability band and hurts the strongest agents. Across five models spanning 20–53% unaided resolution, the three at or below 31.3% never lose from the phased document, gaining 4.0, 4.1, and 1.0 points, while the two above 47% lose 2.0–9.1 points under every memory condition, with Opus 4.8 falling from 47.5% to 39.4%. The oracle document preserves the pattern, so the reversal is a property of the recipient, not a retrieval failure. Haiku 4.5 gains 4.0 points from the document alone but only 1.0 with the record added.

• Simpler retrieval suffices. Cosine similarity over embeddings already selects the labeled trajectory for 62.6% of tasks with the discovered schema, 7 points above LangMem before any rerank; reranking raises this to 68.7%, realizing 87.2% of first-stage recall. REM recovers the labeled trajectory within five candidates for 78.8% of tasks, where LangMem needs ten for 73.3%.

• The extraction schema can be learned rather than written. Automated schema discovery over nine example trajectories, one token-count proxy, and up to five revision iterations moves match@1 from 66.7% (hand-written schema) to 68.7%, so porting to a new domain needs example trajectories rather than prompt engineering.

Methodology in Plain English

Benchmark and agent. The authors evaluate on SWE-ContextBench Lite, which provides 99 repository-level bug-fixing tasks, a pool of 300 solved trajectories, and a human label marking which trajectory is related to each task — this label lets them separate retrieval quality from the effect of the retrieved trajectory. Every condition runs the same agent (Claude Sonnet 4.5 by default, varied across five models later) built on the Strands framework with six tools (bash, read, write, edit, grep, glob) in a per-task Docker container, capped at 100 turns. A run counts as resolved only if the benchmark's held-out tests pass on its patch, scored intention-to-treat over all 99 tasks.

Write path (what to store). Each stored trajectory is a transcript of 55K–170K tokens. The system flattens it to text — keeping user text, assistant text, and tool-call inputs, dropping tool results — which shrinks it roughly fifteenfold. One LLM call then extracts typed fields (files, functions, root cause, fix approach, tests), and a deterministic renderer lays them out as a four-phase document organized by localize, understand, fix, and verify. The result averages about 270 tokens, two orders of magnitude smaller than the transcript. The same document serves as both the retrieval unit and the injected context.

Read path (what to search). The pool and the new task's problem statement are embedded with an off-the-shelf embedding model (Amazon Titan Text Embeddings v2), the five nearest documents by cosine similarity go to a single LLM rerank call, and the selected document is placed in a file the agent is instructed to read first. The reranker never abstains; an unparseable reply falls back to the nearest document by cosine similarity.

In-loop recall and forgetting. A second tier — the implementation record, meaning the raw edits and verifications from the same trajectory, capped at 12,000 characters — is withheld until the agent genuinely fails a validation (edits to test files are filtered out so the agent's own test-writing does not count as struggle). After one exposure it is evicted; the phased document stays in context for the whole run.

Schema discovery. A meta-loop learns the extraction schema: an LLM writes a candidate prompt, an LLM judge scores each extraction from 1 to 5 on coverage, specificity, discriminability, conciseness, and structural consistency, and weak dimensions are revised for up to five iterations, using nine trajectories from the pool (none of them an evaluation task).

Measurement and statistics. To test whether agents actually engage with what they receive, an offline probe extracts identifiers (file paths and CamelCase or snake_case names of at least four characters) from the exact injected text and records which appear in the agent's own text or tool-call inputs, discounting identifiers the paired no-memory run already surfaced. Resolution contrasts are paired by task and tested with the two-sided exact McNemar test pooled over replicates, with a per-task Wilcoxon signed-rank check; utilization measures use Cohen's d and the two-sided Mann-Whitney test. REM, no-memory, and the recall ablations were run three times (297 pooled task-runs); most other conditions were run once and read as directional. Cost is weighted by token type (fresh ×1, cache write ×1.25, cache read ×0.1, output ×5).

Why This Matters

Impact on research. The paper challenges the dominant assumption that experience reuse fails because the right past run is hard to find. It shows that an oracle-retrieved experience injected naively can leave resolution unchanged, and that the stored form and lifecycle governance determine how much of the memory an agent actually uses. Framing memory as three separable, individually measurable decisions — consolidate, surface, retire — gives the field a unit of analysis and suggests these are exactly the operations a continually learning agent must eventually automate over its own growing experience.

Real-world applications:

• Enterprise coding assistants that work repeatedly against an evolving codebase, where accumulated debugging experience is a standing asset rather than a one-off.
• Automated incident response and root-cause analysis, whose work follows the same localize-understand-fix-verify structure REM assumes.
• Text-to-SQL repair and data-pipeline debugging, where the authors note the discovered schema already ports.
• Cost- and latency-sensitive agent deployments, where cutting cost-weighted tokens by 17% and mean wall-clock time from 11.9 to 8.8 minutes per task matters at scale.

Industry relevance. Every gain here is a cost or reliability gain an operator cares about: fewer tokens, fewer turns, fewer unappliable patches (7.0 to 4.0), and less exposure of credentials or personal data captured in agent traces because a ~270-token document re-enters context instead of a raw run. The finding that memory helps mid-capability agents but costs the strongest ones 2.0–9.1 points is directly actionable: the authors suggest a short no-memory pilot per model to decide when to attach memory at all.

Future Directions

• From single-pass reuse to compounding memory. REM currently reuses a fixed pool within a single new run. Letting an agent's own completed runs re-enter the store — with staleness and contradiction handled — is the stated path toward memory that compounds over time and toward continual learning for agents.

• A better struggle signal. The trigger is a single hand-written observable (the first genuinely failed validation). A calibrated trigger drawing on richer evidence, such as repeated failed edits or lack of progress, and spending a bounded disturbance budget only on runs that would otherwise have succeeded unaided, is a clear next step.

• Stricter forgetting. Eviction removes the implementation record, but the agent's own response to it remains in the history; a stricter variant could prune that response too.

• Better measurement and broader validation. The utilization probe establishes presence by substring match rather than causal use, inviting richer attribution. Beyond that: re-running existing memory systems through a shared embedder and reranker to isolate the stored unit, measuring how closely labeled trajectories' patches resemble gold fixes and how stored fixes age as codebases evolve, and validating the full pipeline end-to-end in a second domain such as text-to-SQL.

Target Audience

Researchers and engineers building LLM agents that operate over long horizons — particularly coding, debugging, and software-maintenance agents — who are designing memory, retrieval, or context-management layers. It is also valuable for practitioners deploying agents repeatedly against evolving codebases, who need to reason about token cost and context placement, and for continual-learning researchers interested in how experience should be consolidated and retired rather than merely retrieved. Readers need some familiarity with agent loops and benchmark evaluation, but the argument is accessible to anyone who has watched an agent waste turns re-exploring a problem a previous run already solved.

Authors’ abstract

Large language model (LLM) agents now autonomously complete long-horizon tasks, such as repository-level coding, through hundreds of think-act steps, yet the experience each run produces is rarely reused on the next. Prior memory systems concentrate on retrieval, finding the most relevant past trajectory. Retrieval cannot observe the running agent, so it cannot decide whether the agent needs an experience, at which point, or for how long. We introduce REM (Reusable Experience Memory), a framework that manages the memory lifecycle from inside the agent's think-act loop through three explicit decisions. What to store: each completed trajectory is consolidated into a phase-structured document, two orders of magnitude smaller than the trajectory. When to recall: the document is injected at the start of the task; the costlier implementation record is withheld until an in-loop gate detects the agent struggling. What to forget: the record is evicted after a single exposure. On repository-level bug fixing (SWE-ContextBench Lite), REM retrieves the related trajectory more accurately than every state-of-the-art memory system (68.7% vs. 55.6% match@1) and improves, in every replicate, end-task resolution (27.6% vs. 24.9%) at 17% lower cost. Injecting the same experience as a summary or raw trajectory, even when oracle-retrieved, does not. Ablating any of the three decisions reduces resolution. Altogether, these results demonstrate that the bottleneck in experience reuse is not retrieval fidelity but how experience is stored and governed across its lifecycle, opening a control-theoretic perspective on agent memory.