EvoMemState: Evolving Memory State for Personal Agents

AI summary

Overview

Research area: Long-term memory for large language model (LLM) personal agents (cs.IR), sitting at the intersection of memory systems, retrieval, user modeling, and reinforcement learning.

Technical level: Intermediate. The paper is readable with a general grasp of LLM agents and retrieval, though Section 3.3 assumes familiarity with policy-gradient / GRPO-style reinforcement learning.

Scope: The paper proposes EvoMemState, a "state-centric" memory framework that augments stored personal facts with evolving relational states and a request-specific reasoning state, and evaluates it on four long-term memory and memory-use benchmarks.

What This Paper Is About

Existing personal-agent memory systems (Mem0, MemoryBank, A-MEM, MemGPT, LightMem, and others) are fact-centric: they extract, store, update, and retrieve individual factual records. The authors argue this misses two things — how a new fact updates, constrains, or conflicts with earlier facts, and how retrieved facts should actually be applied to the current request. EvoMemState addresses both by keeping the original facts but attaching two complementary states: Relational States (is a fact still valid, what is its scope, how does it relate to other facts) and a Reasoning State (how should these facts be used, with what role and priority, for this particular request).

Key Contributions

1. Diagnosis of the fact-centric paradigm. Two preliminary studies on the Valid Memory Selection subset of MemSyco-Bench show that related memories are often not retrieved together, and that even complete retrieval does not guarantee correct use.
2. A state-centric memory formulation. Personal memory is represented as Persistent Facts plus Relational States plus a Reasoning State, formalized as a Memory State S_t(q_t) = (R_t, Z_q_t).
3. Three coupled components: Ontology-Guided Memory State Induction (builds a personal memory ontology and derives Relational States), RL-Based State Evolution (trains a Memory State Evolution Policy with GRPO to generate Reasoning States), and State-Aware Reasoning (relation-aware retrieval plus state-grounded response generation).
4. Empirical evaluation on MemSyco-Bench, MemTrapBench, LoCoMo, and LoCoMo-Refined against nine baselines, with ablations, cross-system transfer tests, and retrieval-completeness analysis.

Main Findings

• Fact-centric systems miss related memories. MemoryBank, LightMem, and Supermemory failed to retrieve all relevant information in 58.9%, 61.8%, and 58.0% of cases respectively. For four of five evaluated systems, answer accuracy was 5.9–33.3% higher when all required memories were retrieved.
• Complete retrieval is not enough. A fixed self-reflection prompt reduced accuracy for most systems, with the largest drop of 1.6% on Mem0, and lowered average accuracy from 50.5% to 48.9%.
• Best MemSyco-Bench accuracy. EvoMemState reached 57.6% (Qwen3-8B) and 58.9% (GPT-4o-mini), outperforming the strongest baselines by 12.2% and 5.9%. The paper reports best When to Use and How to Use accuracy, lowest sycophancy rate, and highest Correct Use with Qwen3-8B.
• Best LoCoMo-Refined judge scores. EvoMemState achieved the highest LLM-judge and token F1 scores with both backbones, beating the strongest baseline in judge score by 6.1% and 7.0%.
• Best safety and task-boundary scores on MemTrapBench, indicating that reasoning states prevent irrelevant history and prior task rules from overriding the current request.
• Strong complete retrieval rate. EvoMemState reached 85.6% complete retrieval, described as comparable to the best baseline.
• Large gains on complete-evidence cases. Acc@complete was 70.8% on LoCoMo-Refined and 68.5% on MemSyco-Bench, exceeding the strongest baselines by 11.5% and 31.2%.
• Reasoning states transfer across memory systems. All six baselines improved when paired with generated reasoning states, with gains of 0.2% to 4.9%; NaiveRAG (+4.9%), MemoryBank (+4.8%), and A-MEM (+3.4%) benefited most.
• Ablation (Qwen3-8B). Full model overall score 59.9; removing the Relational State gives 55.4, removing the Reasoning State gives 53.0, and removing RL-based evolution gives 53.2. Removing the Relational State costs 8.9% on MemSyco-Bench and 6.4% on MemTrapBench but only 2.5% and 0.3% on LoCoMo-Refined and LoCoMo. Removing dynamic rules costs 8.0% and 15.0%; generating rules without RL gives similar losses of 7.9% and 16.4%.

Methodology in Plain English

1. Turn conversations into time-ordered facts. A model (DeepSeek-V4-Flash) reads the accumulated dialogue and extracts Persistent Facts, each a single proposition about a preference, experience, belief, circumstance, or constraint, with its source and timestamp.
2. Grow a personal memory ontology. Facts are inserted one at a time into a graph whose nodes are concepts and facts. Hierarchy and attachment edges use PARENT_OF and CONTAINS, with CREATE, MERGE, and SPLIT revising the concept tree. Relation edges use SUPERSEDES for updates and CONFLICTS_WITH to keep unresolved contradictions rather than erasing them. Each fact keeps its original content; only its validity, scope, and relations are updated.
3. Read off Relational States. From the graph, each fact gets a state (fact, validity/scope, relations), so a retrieved record carries the context needed to interpret it.
4. Teach a small model to write request-specific guidance. A Memory State Evolution Policy (initialized from Qwen3-4B-Instruct-2507) is trained with GRPO. For each training request, it samples several candidate Reasoning States; each is fed to a frozen personal agent along with the same memory evidence; the reward is the score difference versus answering with the evidence but no Reasoning State. Group-normalized advantages plus a KL penalty update the policy, so it learns from whether its guidance actually improved answers.
5. Retrieve by traversal, not just similarity. At inference, K semantically similar facts are used as seeds and the graph is traversed in both directions to pull in connected updates, conflicts, and constraints (duplicates removed).
6. Answer with both states. The policy writes a Reasoning State naming applicability, role, and priority; the response model (Qwen3-8B or GPT-4o-mini) answers using facts, Relational States, and that Reasoning State.

Why This Matters

Research impact. The paper reframes personal memory from "store and retrieve facts" to "maintain and evolve the state of memory," and shows empirically that retrieval completeness and memory-use correctness are separable problems. The finding that a fixed self-reflection prompt hurts accuracy while learned state generation helps is a concrete argument against hand-written memory heuristics, and the cross-system transfer result suggests reasoning states could be a drop-in layer over existing memory backends.

Real-world applications.

• Consumer assistants that must respect changed preferences (diet, travel, budget) without forgetting the old ones when answering historical questions.
• Health and fitness coaching, where an old injury should constrain today's recommendations but not affect unrelated requests.
• Customer-support and CRM agents that must resolve conflicting account details rather than silently trusting the newest or the most similar record.
• Long-running task agents that need to know which earlier instructions still bind and which have been superseded.

Industry relevance. Reasoning States are natural-language text passed alongside retrieved evidence and require no change to the response model's parameters, so the approach is compatible with existing deployed backbones. The scaffolding is also inexpensive by the paper's account: memory construction is offline, and the trainable component is a 4B policy rather than the serving model.

Future Directions

1. Multimodal memory. The authors explicitly flag text-only interaction histories as a limitation and call for extending the state representation to documents, images, and other user artifacts.
2. How far do reasoning states transfer? The paper shows gains of 0.2–4.9% when paired with six other memory systems; whether the policy generalizes to memory backends and domains outside those tested is open.
3. Richer relation semantics. The ontology currently uses SUPERSEDES and CONFLICTS_WITH plus structural edges; whether more relation types or automatic conflict resolution would help is not addressed.
4. Cost and latency of state construction at scale. The paper points to an efficiency analysis in its appendix, but the trade-off between ontology traversal, policy sampling, and answer quality is a natural next question for deployment.

Target Audience

Researchers and engineers working on LLM agent memory, personalization, and retrieval-augmented generation; practitioners building long-running assistants that must handle preference drift and contradictory user information; and readers interested in applying reinforcement learning to intermediate reasoning artifacts rather than to final answers. Readers without RL background can still follow the problem framing, the ontology design, and the empirical results.

Authors’ abstract

Long-term memory enables large language model (LLM) agents to provide continuous personalized assistance by retaining user information across interactions. Existing systems support this process through fact extraction, organization, updating, and retrieval, but largely maintain memories as independent facts. This fact-centric paradigm may overlook how new facts update, constrain, or conflict with earlier ones, while leaving their applicability, role, and priority for the current request implicit. In this paper, we introduce EvoMemState, a state-centric framework that augments persistent facts with two complementary states: Relational States that capture how facts remain valid and relate to one another as memory evolves, and a Reasoning State that determines how these facts should be applied to the current request. Specifically, EvoMemState consists of three key components: (i) Ontology-Guided Memory State Induction extracts Persistent Facts and constructs Relational States that maintain their validity, scope, and evolving relations. (ii) RL-Based State Evolution internalizes memory-use experience into a policy through reinforcement learning to generate a request-specific Reasoning State. (iii) State-Aware Reasoning then combines the relevant facts with their Relational States and the generated Reasoning State to guide the personal agent. Extensive experiments on long-term dialogue and memory-use benchmarks demonstrate consistent improvements over existing memory systems in retrieval completeness and memory-grounded response accuracy. Our code and implementation details are available at https://github.com/XMUDeepLIT/EvoMemState.git.