Large Language Model Agents Are Not Always Faithful Self-Evolvers

AI summary

Overview

• Research area: Natural Language Processing — specifically self-evolving LLM agents, agent memory, and the faithfulness/interpretability of model behavior.
• Technical level: Advanced. The work combines causal interventions, four agent frameworks, multiple benchmarks, and layer-level attribution analysis.
• Scope: The paper systematically tests whether self-evolving LLM agents actually use the experience they retrieve, by perturbing that experience and measuring whether agent behavior changes.

What This Paper Is About

Self-evolving LLM agents are designed to improve over time by storing past interactions and reusing them as context for future decisions, without updating the underlying model weights. The paper asks a simple but unexamined question: when an agent is handed experience, does it truly rely on that experience to make decisions, or does performance improve for other reasons? The authors define "experience faithfulness" as the causal dependence of an agent's decisions on the experience it receives, and they measure it by deliberately damaging or removing that experience to see whether behavior changes.

Key Contributions

1. The first systematic investigation of experience faithfulness in self-evolving LLM agents. The authors formalize faithfulness as the causal dependence of an agent's decisions on its input experience, and frame two research questions around whether performance gains are attributable to experience use and why condensed experience is often neglected.
2. A suite of controlled causal interventions for two types of experience. For raw experience (concrete historical trajectories) they design Empty, Shuffle, and Irrelevant interventions; for condensed experience (abstract summaries and heuristics) they design Empty, Corrupt, Irrelevant, and Filler interventions. Each targets a different aspect of the input while preserving surface structure.
3. A large-scale empirical sweep. The abstract reports evaluation of four representative frameworks (ExpeL, Dynamic Cheatsheet, ReasoningBank, G-Memory) across 13 LLM backbones and 9 environments, spanning offline and online paradigms and single-agent and multi-agent settings. The experimental setup section describes the backbones as 10 LLMs, naming closed-source models (GPT-4o(-mini), GPT-5.2, Gemini-2.5-Flash, Gemini-3-Pro, Claude-Sonnet-4.6) and open-weight Qwen3 variants from 1.7B to 32B dense, plus 30B-A3B and 235B-A22B MoE models.
4. A three-part causal explanation of the faithfulness gap, traced to the experience itself, the backbone model, and the task regime, supported by an error taxonomy, Integrated Gradients attribution over prompt segments, and additional multi-hop QA experiments.

Main Findings

• Agents are strongly and reliably faithful to raw experience. In both the offline ExpeL framework and the online Dynamic CheatSheet framework, removing raw experience caused substantial performance degradation on most tasks, and perturbing raw trajectories (especially Empty or Irrelevant replacements) produced similarly severe declines.
• Condensed experience often has minimal behavioral influence. Interventions such as Corrupt, Irrelevant, and Filler on condensed experience yielded performance nearly indistinguishable from the unperturbed baseline. Even removing condensed experience altogether had only marginal impact when raw experience was also present.
• The asymmetry holds across offline and online paradigms. Despite different mechanisms for accumulating experience, ExpeL and Dynamic CheatSheet showed strikingly similar faithfulness patterns, suggesting the gap is a property of current self-evolving designs rather than one memory-update strategy.
• The problem persists when condensed experience is the only input. In ReasoningBank, which provides no raw trajectories, removing condensed experience still caused consistent performance drops across all four WebArena tasks, but semantic perturbations (Corrupt, Irrelevant, Filler) led to only negligible degradation or even slight improvements. This rules out the idea that raw experience is merely overshadowing condensed experience.
• The asymmetry holds in multi-agent settings. In G-Memory, perturbing either Reference Raw Experience (curated offline) or Execution Raw Experience (collected during operation) consistently degraded performance on ALFWorld, while condensed experience perturbations had only marginal effects.
• Scaling model size does not fix it. Across six Qwen3 variants from 1.7B to 32B parameters under ExpeL, larger models achieved higher unperturbed success rates but remained markedly more faithful to raw experience than to condensed representations. Since ExpeL's condensed summaries are generated by the backbone itself, the authors argue the unfaithfulness is not solely a matter of summary quality.
• Task type modulates sensitivity. On knowledge-intensive benchmarks such as GPQA-Diamond, MMLU-Pro Eng., and FEVER, raw and condensed experience had more comparable influence, and the effect of raw-experience perturbation was less pronounced.
• Condensed experience can actively hurt. When the agent succeeded without condensed experience but failed with it, failures clustered into three modes: distraction from the task goal (e.g., 45.2% for Gemini-2.5-Flash on WebArena Shopping), overreliance on incorrect priors, and premature inference from prior patterns (e.g., 50.0% for Gemini-2.5-Flash on Map; 39.1% for Qwen3-14B on Reddit).
• Internal attribution confirms underutilization of condensed content. Layer-wise Integrated Gradients under ExpeL with Qwen3-32B, segmenting prompts into System Instruction, Condensed Experience, Raw Experience, and Current Trajectory, showed consistently low attribution for condensed experience across layers, moderate and stable attribution for raw experience, and dominant attribution for the current trajectory in later layers.
• On multi-hop QA with strong pretrained priors, experience had limited causal influence. With ExpeL and Qwen3-32B over 100 sampled examples each from 2WikiMultiHopQA and Musique (exact match), the baseline scored 62 and 48 respectively, and neither raw nor condensed interventions produced consistent degradation.

Methodology in Plain English

The researchers treat the agent like a black box and ask what happens when you damage its inputs. First, they define two kinds of memory: raw experience, which is full recorded trajectories of what an agent did on similar tasks, and condensed experience, which is distilled advice such as heuristics or general lessons. Then they build interventions that target one type at a time. For raw experience, they strip out the content while keeping the prompt wording ("Empty"), scramble the order of steps inside a trajectory ("Shuffle"), or swap in trajectories from unrelated tasks ("Irrelevant"). For condensed experience, they empty it, corrupt key details, replace it with unrelated advice, or fill it with meaningless placeholder characters. If changing the experience changes the agent's behavior, the agent is judged to be faithful to it; if the agent behaves the same regardless, it is not.

They run these interventions on four published agent frameworks — ExpeL (offline, single-agent), Dynamic Cheatsheet and ReasoningBank (online, single-agent), and G-Memory (online, multi-agent) — using each framework's official configuration and a frozen LLM backbone, so any behavioral change must come from the external memory. They cover 9 benchmarks across knowledge QA, math reasoning, embodied action, and web interaction, and repeat the setup across multiple models, including closed-source and open-weight Qwen3 models at several scales. To explain the results rather than just report them, they inspect failure cases that arise only when condensed experience is added, and they use Integrated Gradients to measure how much each prompt segment contributes to the model's output across attention heads and layers.

Why This Matters

Impact on research. The paper challenges a common assumption behind memory-based agent design: that accumulating and reusing experience necessarily means the agent is conditioning on it. The authors argue that performance gains can arise without faithful grounding — for example, from the mere presence of a text block or stylistic patterns rather than the actual semantic content of condensed summaries. This reframes how self-evolving methods should be evaluated, and supplies a reusable intervention-based protocol for doing so.

Real-world applications (implications drawn from the paper's settings):

• Web-interaction agents that shop, browse, or operate CMS and forum platforms, where the paper shows condensed guidance can push agents toward assumptions that do not match the live page.
• Embodied and household-task agents such as those tested in ALFWorld, where both curated and self-collected raw trajectories were faithfully used but condensed summaries were not.
• Knowledge-intensive verification and QA, such as fact-checking claims in FEVER or multi-hop questions, where pretrained knowledge alone may already be sufficient and retrieved experience adds little.
• High-stakes or safety-critical deployments, which the impact statement explicitly highlights, where the authors argue the gap between reported utility and actual faithfulness raises controllability concerns.

Industry relevance. Teams building agent products with persistent memory, retrieval pipelines, or "self-improving" loops invest significant engineering effort into condensation and summarization of past experience. This paper suggests that condensed memory written as abstract advice may be silently ignored by the backbone, or worse, may degrade behavior — which argues for measuring causal dependence on memory rather than inferring it from end-task scores.

Future Directions

1. Designing condensed experience for usability, not brevity. The authors call for condensed experience that is contextualized, task-relevant, and cognitively actionable rather than abstract summaries or generic advice, and they connect this to automatic condensation methods optimized for alignment and usability.
2. Rethinking the timing and mechanism of experience injection. Instead of statically prepending memory to every input, experience should be retrieved and injected dynamically based on task demands, interaction history, and internal model uncertainty, possibly as an interactively triggered signal.
3. Determining when experience is needed at all. The paper finds that indiscriminate use can dilute attention and reduce effectiveness, and that some tasks are handled by pretrained priors alone — leaving open how to decide when to retrieve experience.
4. Building agents that use experience faithfully as well as effectively. The conclusion frames this as an open goal: mechanisms for experience-driven adaptation that are reliable and interpretable, not merely performance-improving.

Target Audience

This paper is most useful to researchers and practitioners working on LLM agents, agent memory architectures, and self-improving or self-evolving systems. It is also relevant to interpretability and evaluation researchers interested in faithfulness testing beyond chain-of-thought, to engineers building production agents with retrieval or long-term memory, and to teams evaluating the reliability of memory-augmented systems in high-stakes settings. Readers should be comfortable with agent frameworks, benchmark-based evaluation, and causal intervention methodology.

Authors’ abstract

Self-evolving large language model (LLM) agents continually improve by accumulating and reusing past experience, yet it remains unclear whether they faithfully rely on that experience to guide their behavior. We present the first systematic investigation of experience faithfulness, the causal dependence of an agent's decisions on the experience it is given, in self-evolving LLM agents. Using controlled causal interventions on both raw and condensed forms of experience, we comprehensively evaluate four representative frameworks across 13 LLM backbones and 9 environments. Our analysis uncovers a striking asymmetry: while agents consistently depend on raw experience, they often disregard or misinterpret condensed experience, even when it is the only experience provided. This gap persists across single- and multi-agent configurations and across backbone scales. We trace its underlying causes to three factors: the semantic limitations of condensed content, internal processing biases that suppress experience, and task regimes where pretrained priors already suffice. These findings challenge prevailing assumptions about self-evolving methods and underscore the need for more faithful and reliable approaches to experience integration.