# Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents

AI.info

> WikiTrace links each distilled agent lesson to its source trajectory, letting agents catch and reject bad memories instead of blindly following them.

**arXiv**: 2611.14320

**Published**: 2026-11-10

**Authors**: Yukyeong Lee, Yumin Choi, Jinheon Baek, Sung Ju Hwang

## AI summary

## Overview

**Research area:** LLM agents and agent memory — specifically how agents distill past task trajectories into reusable “lessons” (procedural memory) and how that memory can backfire.

**Technical level:** Intermediate. The paper assumes familiarity with LLM agents, ReAct-style loops, tool use, and standard agentic benchmarks, but its core idea (linking a lesson back to the run it came from) is conceptually simple.

**Scope:** The paper diagnoses *memory-induced degradation* in LLM agents, proposes WikiTrace — a three-layer memory (insight → compilation → raw trajectory) that lets an agent re-check a lesson against its source evidence at test time — and evaluates it on eight suites across five agentic benchmarks.

## What This Paper Is About

External memory lets an LLM agent turn past trajectories into concise lessons it reuses on new tasks, but lessons are inferred from runs labeled only with success or failure, so a lesson can be wrong, or misapplied, or misread. The authors show this causes **memory-induced degradation**: the agent fails tasks it would solve with no memory at all, and in 52% of such failures the agent was following a memory item. WikiTrace's goal is to let the agent check each lesson against the trajectory it came from before acting on it, so memory produces more gains without breaking tasks the agent already handles.

## Key Contributions

1. **A measurement of memory-induced degradation.** The authors separate aggregate accuracy into **Gain** (rate at which memory turns prior failures into successes) and **Regression** (rate at which it turns prior successes into failures), which average accuracy hides.
2. **A failure analysis of degradation.** Using an LLM judge (`gpt-5.6-sol` at medium reasoning effort) on tasks the agent solves in all three no-memory trials but fails with memory, they find 52% of failures trace to a memory item the agent followed, most often a wrong item (20%); others involve a valid item applied outside its scope, misused, or followed while memory held a better one.
3. **WikiTrace, an evidence-linked memory system.** Each lesson is stored as a concise *insight* linked to a *compilation* (a diagnosis of why its run succeeded or failed) and to the *raw trajectory* itself; all layers are indexed, mounted read-only, and searched with `read` and `bash`.
4. **A test-time descent mechanism.** The agent starts from all insights and descends through the links when an insight lacks details needed to act, conflicts with another insight, or comes from a failed run — and again whenever an action fails.

## Main Findings

- **Best on every suite:** WikiTrace averages **90.2** across the eight suites versus **79.5** for No Memory (a 10.7-point gain) and **87.0** for Insights Only, a baseline given the identical insights but no source compilations, raw trajectories, or descent prompt (a 3.2-point gap).
- **It is the only memory method that matches or exceeds No Memory on every suite.** AWM averages 81.6, ReasoningBank 81.7, and ExpeL 80.3.
- **Lower regression:** WikiTrace's Regression is **4.5%**, versus **6.5%** for Insights Only and **9.7%** for a rerun baseline of three additional No Memory trials. It is lower than Insights Only on seven of eight suites and is the only method whose confidence interval lies entirely below the rerun band.
- **Higher gain:** WikiTrace exceeds the rerun baseline by **35 percentage points** on Gain, compared with 27 for Insights Only and 7 to 14 for existing memory methods.
- **Half as many bad-item regressions:** relative to Insights Only, wrong or out-of-scope items cause half as many of WikiTrace's regressions.
- **Descent scales with difficulty:** the agent descends in **22%** of runs on Easy tasks and **44%** on Hard ones. Descent runs lead Insights Only by **1.6 to 4.7 points** more than Stay runs at every difficulty and on every text-world benchmark; the Descent–Stay gap grows from **1.8 points** (Easy) to **3.7** (Medium) and **4.1** (Hard).
- **More selective memory use:** even in Stay runs, WikiTrace reports **19 to 28%** fewer insights as used than Insights Only, and **38 to 57%** fewer in Descent runs.
- **Descent pays off after problems, not routine checks:** descents following a blocked attempt, a blocking observation, or a doubt make up **36%** of descents and lead Insights Only by **10.4 points**, versus **3.7 points** when nothing has gone wrong. The largest lead, **17.3 points**, comes when a blocked attempt sends the agent after a specific fact. Routine checks are the most common descent (**38%**) and yield a **3.4-point** lead.
- **Evidence helps most when it changes the plan:** the opened run changes the agent's plan in **43%** of descending runs (revising an insight 24%, refining it with a missing detail 14%, supplying a new fact 4%), and those runs lead Insights Only by **8.3 points** versus **4.4** when the evidence only confirms the plan or goes unused.
- **Failed runs are useful but risky alone:** on PlanCraft and ScienceWorld, both methods do better with failed-run insights alone than successful-run insights alone and best with both. Insights Only reaches its highest Regression with failed runs alone (9.3% on PlanCraft, 9.4% on ScienceWorld); WikiTrace cuts these to **7.1%** and **8.9%**.
- **Not a budget artifact:** giving AWM 9 build runs and ExpeL and ReasoningBank 3 build runs does not overtake WikiTrace, which still has the highest average.
- **Modest test-time overhead:** **1.17×** the per-task cost of Insights Only, for a 3.2-point average improvement.
- **Holds on a second backbone:** with `GLM-5.3-flash`, WikiTrace averages **92.7**, beating Insights Only by 1.8 points and No Memory by 21.5, and is best on all three evaluated suites; every existing memory method falls below No Memory on at least one suite.

## Methodology in Plain English

The setting is an offline corpus of past runs, each consisting of a task, a full trajectory of actions and observations, and a binary success/failure label. No gold solutions or expert explanations are available, and memory must be built once and frozen — no extra runs to validate individual memory items.

**Building memory.** A compiler reads each run and produces two things: a *compilation* (a short diagnosis of the key decisions and observations behind the outcome, using separate templates for successes and failures) and at most three *insights* (each stating what to do, when it applies, which run it came from, and whether that run succeeded). The raw trajectory is preserved unchanged. Every insight links to its compilation and its raw trajectory. Insights are never updated or deleted, so the pool may contain redundant or conflicting items — resolving that is left to the agent at test time.

**Using memory.** At test time the agent first sees all insights, plus the three past runs whose tasks are most similar to the current one. The memory is mounted read-only as files, and the agent can descend from an insight into its compilation or raw trajectory using terminal tools (`read`, `bash`) when the insight lacks needed detail, conflicts with another, or comes from a failed run — and after any failed action.

**Evaluation.** Eight suites across five benchmarks: ALFWorld, ScienceWorld, and PlanCraft (text worlds), plus τ³-bench Retail and AppWorld (tool use). Each trial builds memory from the same No Memory training runs and tests on the same tasks, with three trials per suite. The agent is a Pi terminal agent; the agent and memory builders use `deepseek-v4-flash` with high reasoning effort. Baselines are No Memory, AWM, ExpeL, ReasoningBank, and the Insights Only ablation. Analysis of *why* the agent descends uses LLM judges, with `gpt-5.6-sol` and `GLM-5.3` agreeing at Cohen's κ of 0.77 on trigger and 0.63 on target; effect labels are the majority of three judges.

## Why This Matters

**Impact on research.** The paper reframes memory quality as a question not only of *which* lessons to store but of whether an agent can *justify* a lesson when applying it. Splitting accuracy into Gain and Regression gives the field a way to report harmful memory effects that average scores conceal, and the finding that 52% of degradations trace to a followed memory item (20% to outright wrong items) makes the problem concrete.

**Real-world applications:**

- **Customer-service and enterprise agents** that accumulate playbooks over time, where a stale or out-of-scope rule can break a request the agent previously handled.
- **Software and DevOps agents** that reuse procedures across repositories or environments and need to check a past command format or log against the run it came from.
- **Robotics and simulated-environment planners** (crafting, household, scientific-experiment tasks) where a subtly wrong procedural rule silently blocks an otherwise solvable task.
- **Continual-learning deployments** where memory is frozen and audited, so traceability from a lesson back to its source execution is a compliance requirement as much as a quality one.

**Industry relevance.** The overhead is modest (1.17× test-time cost for a 3.2-point average gain) and the benefit survives larger construction budgets for competing methods and a second backbone model, which matters for teams that cannot afford to re-run validation tasks for every memory item.

## Future Directions

- **Retrieval instead of full injection.** WikiTrace puts every insight in the first message so none is missed, so the prompt grows with memory; a memory too large to place in full needs a retrieval mechanism, which the authors explicitly leave open.
- **Verification rather than inspection.** A source link lets the agent check an insight against its run but does not verify it; stronger admission or revision of insights from binary outcomes remains unsolved.
- **Better failure diagnosis.** Compilations are interpretations of a run labeled only success or failure, so the diagnosis and any insight drawn from it may be wrong — richer outcome signals could tighten this.
- **Moving beyond LLM-judged analysis.** Both the degradation-cause labels and the descent labels come from judges reading the agent's reasoning, so they describe what that reasoning states, not what produced the outcome; more direct causal measurement is an open problem.

## Target Audience

Researchers and engineers building LLM agents with persistent memory, especially those working on experience reuse, procedural memory, and continual learning for agents. It is also useful for practitioners deploying agents where memory silently breaks previously working tasks, and for readers interested in evaluation methodology — the Gain/Regression split is directly reusable even by those who do not adopt WikiTrace itself.

## Authors’ abstract

External memory enables LLM agents to improve from experience by distilling trajectories of past tasks into concise lessons for new tasks. However, memory can also lead to memory-induced degradation, where an agent fails on tasks it would solve without memory, often because it follows a lesson that is incorrect, or misapplies or misreads a valid one. This is because lessons are inferred from trajectories labeled only with task success or failure, yet are typically provided…

[The full abstract on arXiv](https://arxiv.org/abs/2611.14320)

[Read the original paper](https://arxiv.org/abs/2611.14320)

---

Canonical: https://ai.info/paper/don-t-just-trust-your-memory-evidence-linked-memory-reduces-memory-2611.14320
