A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents

AI summary

Overview

Research area: Natural Language Processing; specifically the memory systems of LLM-based autonomous agents (retrieval-augmented generation, long-context modeling, personalization, continual learning, reflection, and agent evaluation).

Technical level: Intermediate. The survey is conceptual and taxonomy-driven, but it assumes familiarity with LLM inference, retrieval-augmented generation, and basic agent loops.

Scope (one sentence): A large multi-institution survey that organizes hundreds of papers on foundation agent memory along three axes — substrate, cognitive mechanism, and subject — and argues that memory has shifted from passive storage to the substrate through which agents self-evolve in long-horizon, real-world settings.

What This Paper Is About

The paper argues that AI research has entered a "second half" in which model architecture and benchmark scores matter less than real utility in long-horizon, dynamic, user-dependent settings such as agentic coding, deep research, and computer use. In these settings, LLM-based agents face context explosion beyond fixed context windows and must continuously accumulate, manage, and selectively reuse large volumes of information across extended interactions. The survey's goal is to unify a fragmented literature — RAG, long-context modeling, cognitive memory architectures, personalization, continual learning, reflection, and agent evaluation — into one taxonomy, and to position memory as the mechanism that closes the gap between idealized benchmark performance and real-world deployment.

Key Contributions

1. A three-dimensional taxonomy. The authors connect three dimensions that prior reviews typically treat separately: memory substrate (what form memory takes — internal versus external), memory cognitive mechanism (how memory functions — sensory, working, episodic, semantic, procedural), and memory subject (who memory serves — user-centric personalization versus agent-centric experience).

2. A system-design analysis of memory operations. The survey analyzes how memory is instantiated and operated under single-agent versus multi-agent topologies, distinguishing fundamental memory operations in single-agent systems from memory routing in multi-agent settings.

3. Memory management as a learned capability. The authors highlight learning policies over memory operations, showing how memory management itself is becoming trainable through reinforcement-learned context curation and experience consolidation at decision time, plus an emerging ecosystem of explicit, portable, shareable agent skills surfaced through agent harnesses, context engineering, and standardized tool-mediation protocols.

4. Positioning memory as the engine of self-evolution, with six open challenges. The paper frames memory as the central mechanism for real-world utility and agent self-improvement, and outlines six open challenges to guide future work. (The provided content states the number of challenges but does not enumerate them individually.)

Main Findings

• Publication growth is steep and concentrated in 2025. From an initial pool of several hundred papers, the authors curated 218 key articles published between 2023 Q1 and 2025 Q4. Cumulative volume rises sharply throughout 2025, with the steepest growth in Q4. Figure 3 partitions these 218 papers by substrate and by subject (both partitions sum to 218), while the cognitive-mechanism panel sums to more than 218 because a single paper may exhibit several mechanisms.

• Substrates split into external and internal. External memory stores information outside the model's parameters or state using vector indices, text records, structural stores (relational tables, graph-based, tree-based), or hierarchical multi-store designs with a meta-memory manager. Internal memory covers weights (parametric memory), latent states, and KV caches.

• External memory trades latency and noise for scalability and update flexibility. It permits cross-session retention and continual updates without retraining, but retrieval can be unreliable — poorly developed similarity or ranking algorithms return irrelevant results that introduce noise and reduce agent utility. Storage and indexing costs escalate as memory grows, so a well-built external store needs summarization, selective retention, forgetting, deduplication, and periodic pruning.

• Internal memory trades update cost and interference risk for fast recall. Weight memory recalled through the forward pass avoids external retrieval, but modifying weights is computationally costly and may introduce interference, overwriting, or distortion of previously learned knowledge. Latent-state memory supports within-session coherence but is generally not durable across sessions unless exported externally, and its resource cost scales with model depth, batch size, and precision.

• Weight updates follow three strategies. The survey groups them as continual learning (incremental updates that attempt to avoid catastrophic forgetting), model editing (localized factual associations, including rank-one updates to decisive mid-layer weights and small multi-layer updates that can modify thousands of memories), and distillation (compressing context- or prompt-dependent behavior into parameters).

• KV cache compression is an active lever. Representative approaches include dynamically evicting tokens that contribute least to attention scores while balancing "heavy hitters" with recently generated tokens; clustering important features from consistent attention patterns within a prompt's observation window; and ranking key-value vectors by importance and using binary search to allocate per-layer cache sizes under a global budget.

• Eight trade-off dimensions summarize substrate choice. Table 1 compares internal versus external memory on retrieval latency, storage cost, update flexibility, context window overhead, cross-session persistence, scalability, forgetting risk, and retrieval precision. Internal memory shows low retrieval latency (via forward pass or GPU-resident caches) with high storage cost, while external memory's latency varies with index structure, corpus size, and retrieval algorithm.

• A utility gap separates benchmark scores from real deployment. The paper notes that LLMs and agents can achieve over 90% accuracy on benchmarks such as MMLU or MATH, yet most agent evaluation protocols use static, pre-defined rules with short, isolated task settings and no multi-turn, long-term interaction — so they do not reflect inherently long-horizon, long-context, user-dependent reality.

• Prior surveys leave gaps. Earlier work organizes memory by task application or management strategy, or projects AI memory onto human memory through functional analogies and memory lifecycles. The authors argue these approaches do not systematically characterize underlying memory substrates or explicitly model the subject that memory serves. Even recent work organizing memory along forms, functions, and temporal dynamics remains "largely partial," focusing mainly on agent-centric tasks rather than how memory should be designed, optimized, and deployed for users.

Methodology in Plain English

This is a survey, not an empirical study, and its method is literature curation plus conceptual synthesis.

The authors queried Google Scholar with memory-related keywords (for example, agent memory, long-term memory, context management, personalization memory) and manually screened proceedings of major computer science conferences and journals, including top-tier NLP, ML, IR, and AI venues. They started with several hundred papers and iteratively screened down to 218 key articles published between 2023 Q1 and 2025 Q4.

They then sorted those papers along three orthogonal axes. First, by substrate: is the memory held outside the model (vector indices, text records, structural or hierarchical stores) or inside it (weights, latent states, KV caches)? Second, by cognitive mechanism: does the memory play a sensory, working, episodic, semantic, or procedural role? Third, by subject: is the memory about the user or about the agent itself?

On top of that classification, they performed a system-design reading — how memory is operated in single-agent versus multi-agent topologies, how memory operations are increasingly governed by learned policies rather than hand-designed heuristics, how memory designs scale across interaction horizon, environment complexity, and number of interacting agents, and how memory utility is evaluated. They also reviewed real-world applications and distilled open challenges.

Why This Matters

Impact on research. The survey supplies a shared vocabulary and a unifying taxonomy for a literature that the authors describe as fragmented across RAG, long-context modeling, cognitive architectures, personalization, continual learning, reflection, and evaluation. It also reframes the central research question of the "second half": not how to improve training recipes, but how to make agents genuinely useful in long-horizon, dynamic, user-dependent environments. Its claim that memory is the substrate of agent self-evolution — the thread running through cognitive mechanisms, memory operations, and learning policies — gives the field a focal point for measuring whether agents actually improve from their own experience.

Real-world applications in the paper (bullets):

• Agentic coding and computer use, where accumulated project context grows beyond what prompt-based mechanisms can hold.
• Deep research, which requires multi-step execution, tool-augmented decision-making, and reuse of prior findings.
• GUI and web interaction / embodied control in simulated or real environments, where interaction histories are long and environment state evolves.
• Personal health-care, tutoring, workflow automation, and human-agent collaboration, where long-term user history drives personalization.

Industry relevance. The author list spans academic institutions (UIC, IIT, UIUC, UW–Madison, ASU, Emory, Northwestern, NTU, UCF, Rutgers, Cambridge, Harvard, UTokyo, UCSD, UCSC, TAMU, UCSB, MBZUAI, McGill, UCLA, Stanford) and industry labs and companies (Salesforce, Google, Meta, Roblox, Cisco, Capital One). That mix signals that the design questions here — what to store, what to forget, what to expose as shareable skills, and how to evaluate memory utility — are being made in production systems, not just in papers. The survey also points to an emerging ecosystem of explicit, portable, shareable agent skills surfaced through agent harnesses, context engineering, and standardized tool-mediation protocols, which is where cross-vendor interoperability and deployment economics will be decided.

Future Directions

1. Turn memory management into a learned capability. The paper identifies reinforcement-learned context curation and experience consolidation at decision time as an emerging direction, leaving open how such policies should be trained, what reward signals measure memory utility, and how learned policies interact with hand-designed heuristics.

2. Enumerate and address the six open challenges. The paper states that it outlines six open challenges for foundation agent memory; the provided content does not enumerate them, so the specific list is not reported here. Future work should engage with each.

3. Close the user-centric gap in memory design. The paper argues that recent consolidations of agent memory remain largely partial because they focus mainly on agent-centric tasks rather than how memory should be designed, optimized, and deployed for users. Systematic treatment of user-centric memory remains an opening.

4. Build benchmarks that measure memory utility rather than static QA. The authors call for evaluation that moves beyond static question answering toward dynamic, interactive capabilities — tool use, multi-step decision-making, and long-horizon feedback — and the survey reviews benchmarks and metrics for assessing memory utility while stating that designing a benchmark for real environments is one of the most important challenges.

5. Scale memory across horizon, environment complexity, and agent count. Section 6 addresses scalability issues across interaction horizon, environment complexity, and the number of interacting agents and systems — an area where the trade-offs in Table 1 (latency, storage cost, forgetting risk, retrieval precision) are likely to bind hardest.

Target Audience

Researchers and graduate students working on LLM agents, retrieval-augmented generation, long-context modeling, and personalization who need a map of the field rather than a single method. It is also relevant to practitioners designing production agent systems that must persist user state across sessions and long-horizon tasks, and to benchmark designers looking for evaluation criteria that capture memory utility. Readers wanting a purely empirical result or a new model architecture will not find one here; the contribution is conceptual organization, a system-design lens, and an agenda of open challenges.

Authors’ abstract

Research in artificial intelligence is shifting from model innovations and benchmark scores towards problem definition and rigorous real-world evaluation. As the field enters the "second half," the central challenge becomes real utility in long-horizon, dynamic, and user-dependent settings such as agentic coding, deep research, and computer use, where LLM-based agents face context explosion beyond fixed context windows and must continuously accumulate, manage, and selectively reuse information across extended interactions. Memory, with hundreds of papers released in 2025, therefore emerges as the critical solution to fill this utility gap. Beyond passive storage, memory is increasingly the substrate through which agents self-evolve: short-term memory gates which experiences are perceived and abstracted during execution, while long-term memory consolidates them into reusable knowledge and skills, forming the loop through which agents improve from their own experience. In this survey, we provide a unified view of foundation agent memory along three dimensions: memory substrate (internal parametric state and external retrieval-augmented stores), cognitive mechanism (sensory, working, episodic, semantic, and procedural), and memory subject (user-centric personalization and agent-centric experience). We then analyze how memory is operated under single- and multi-agent topologies and highlight learning policies over memory operations, showing how memory management itself is becoming a trainable capability spanning reinforcement-learned context curation, experience consolidation at decision time, and the emerging ecosystem of portable, shareable agent skills. Finally, we review evaluation benchmarks and metrics for memory utility, and outline open challenges and future directions.