REMem: Reasoning with Episodic Memory in Language Agent

AI summary

Overview

Research area: Memory systems for large language model (LLM) agents, with a specific focus on episodic memory (memory of concrete, time-and-place-bound experiences) as distinct from semantic memory (general world knowledge).

Technical level: Intermediate. The paper assumes familiarity with retrieval-augmented generation (RAG), knowledge graphs, embedding models, and ReAct-style tool-using agents.

Scope in one sentence: The paper identifies episodic recollection and episodic reasoning as two progressive challenges for language agents, proposes REMem (a two-phase framework combining a hybrid gist-and-fact memory graph with tool-based agentic retrieval), and evaluates it on four episodic memory benchmarks.

What This Paper Is About

Language agents today store mostly semantic memory: decontextualized facts and concepts, retrieved by embedding similarity. They are poor at episodic memory, meaning the ability to reconstruct a specific past experience along a spatiotemporal axis (when it happened, who was involved, where, and how events relate to one another), and to reason across multiple such events. The paper's goal is to close that gap by formalizing the two capabilities (episodic recollection and episodic reasoning), building a memory representation that explicitly models events and their situational dimensions over time, and giving the agent tool-based, iterative access to that memory during inference.

Key Contributions

1. Formalization of two progressive challenges. The authors identify and formalize episodic recollection (reconstructing events and binding situational elements such as time, location, participant, and emotion to them) and episodic reasoning (multi-step reasoning across a timeline, including event-to-event relations, ordinal constraints, counting, superlatives, and duration estimation).

2. The REMem framework. A two-phase approach: an indexing phase that converts experiences into a hybrid memory graph linking time-aware gists (concise, human-readable event summaries with parsed timestamps) and facts (time-scoped subject–predicate–object triples), and an agentic inference phase that uses a ReAct-style agent with curated tools for iterative retrieval and graph exploration.

3. A hybrid memory graph design with explicit event modeling. Unlike prior work that selectively extracts "important" content or represents memory only as entity relations, REMem instructs the LLM to organize memory primarily along time, grounding gists and facts to a timeline with Wikidata-style qualifiers (point_in_time, start_time, end_time), linking gists to the phrase nodes from the same chunk, and adding synonymy edges between gists whose embedding similarity exceeds a threshold.

4. A comprehensive evaluation over four benchmarks. Experiments on LoCoMo, REALTALK, Complex-TR, and Test of Time against strong embedding baselines, structure-augmented memory systems (Mem0, Graphiti, HippoRAG 2), and the TISER prompting method, plus ablations, refusal analysis, human evaluation, and error analysis.

Main Findings

• Aggregate gains over state of the art. Across four episodic memory benchmarks, REMem shows 3.4% and 13.4% absolute improvements on episodic recollection and reasoning tasks respectively, compared with state-of-the-art memory systems such as Mem0 and HippoRAG 2.

• Episodic recollection results. On LoCoMo (1,986 queries), REMem-I scores 42.4 F1, 32.7 BLEU-1, and 76.2 LLM-J, while REMem-S scores 41.3 F1, 31.5 BLEU-1, and 77.5 LLM-J. On REALTALK (728 queries), REMem-I scores 25.6 F1, 18.1 BLEU-1, and 63.7 LLM-J, and REMem-S scores 26.2 F1, 19.2 BLEU-1, and 65.3 LLM-J. REALTALK, built from real human conversations, is harder than the synthetic LoCoMo.

• Structure-augmented baselines underperform. Mem0 scores 25.1 F1 / 18.0 BLEU-1 / 49.7 LLM-J on LoCoMo and 9.8 / 7.2 / 14.3 on REALTALK. Graphiti scores 33.7 / 28.9 / 52.5 on LoCoMo and 15.1 / 11.5 / 35.3 on REALTALK. HippoRAG 2 scores 39.0 / 30.8 / 74.0 on LoCoMo and 21.9 / 16.2 / 55.8 on REALTALK. The paper attributes these gaps to Mem0 discarding extracted statements by its own decision, Graphiti losing coherent event context by centering on entities, and HippoRAG 2 lacking any temporal or event modeling.

• Episodic reasoning results. On Complex-TR (1,000 queries), REMem-I scores 83.3 F1, 77.6 BLEU-1, 89.6 LLM-J; with TISER it reaches 90.6 F1, 86.0 BLEU-1, 92.0 LLM-J; REMem-S scores 78.5 / 72.7 / 82.6. On Test of Time (2,800 queries), REMem-I reaches 93.1 EM, REMem-S 72.5 EM, and REMem-I with TISER 90.6 EM.

• Only method above 90% EM on Test of Time. REMem-I is described as the unique method exceeding 90% exact match on the Test of Time benchmark.

• Iterative retrieval matters most for reasoning. REMem-I improves over REMem-S by +7.0 LLM-J and +20.6 EM on the reasoning tasks, and improves over Full-Context by +8.0 LLM-J and +13.4 EM, a larger margin than on the recollection tasks.

• Token efficiency. Full-Context consumes an average of 26k input tokens per LoCoMo query, versus 9k for REMem-I and 0.9k for REMem-S during inference.

• Ablations. Removing gists causes the largest degradation (LoCoMo LLM-J drops from 76.2 to 48.9), supporting the claim that gists carry the main situational elements. Removing facts causes a smaller but consistent drop (Complex-TR LLM-J 89.6 to 87.2). Removing synonymy edges reduces F1 and BLEU-1 on both datasets while LLM-J stays nearly unchanged. Removing semantic_retrieve lowers Complex-TR LLM-J from 89.6 to 88.1; removing lexical_retrieve lowers it to 87.5.

• Refusal behavior. Of 1,986 LoCoMo queries, 446 are unanswerable. REMem produces 344 refusals with 73.3% precision, 56.8% recall, and 64.0% F1 (reported as 63.96% in the body text). Graphiti produces 954 refusals with 38.9% precision, 83.6% recall, and 53.1% F1. Mem0 produces 90 refusals with 40.0% precision, 8.1% recall, and 13.5% F1. The authors report REMem improves precision by +34.4 points and F1 by +10.9 points over Graphiti while producing roughly one-third as many refusals.

• Human evaluation supports LLM-as-a-judge. On 100 randomly selected LoCoMo samples, the LLM judge matched human scores in 93% of cases, with only 7 discrepancies. Mean scores were 0.710 for human evaluation, 0.410 for F1, 0.284 for BLEU-1, and 0.740 for the LLM judge. Pearson/Spearman correlations with human judgment were 0.551/0.603 for F1, 0.417/0.531 for BLEU-1, and 0.827/0.827 for the LLM judge.

• Error analysis (LoCoMo, 100 sampled errors). Selection or grounding errors are the most common at 46%, followed by temporal or numerical reasoning mistakes at 19%, and abstentions despite retrieved evidence at 18%.

• Error analysis (Complex-TR, 100 sampled errors). Temporal window mismatch is the most frequent failure at 42%, followed by incomplete or inconsistent multi-entity lists at roughly 21%, offset direction mistakes (confusing before and after) at about 18%, and roughly 5% of cases incorrectly returning "no information available" despite gold facts being present.

• Comparison with plain RAG. REMem outperforms NV-Embed-v2 on questions requiring disambiguation across categories and reconciliation of time-stamped events, while the embedding baseline is reliable on straightforward interval calculations where the answer reduces to simple computation over accurately retrieved facts.

Methodology in Plain English

REMem works in two phases.

Phase 1: Indexing. For each event statement or chat session, an LLM extracts one or more gists: concise, atomic, natural-language sentences capturing the key details (participants, actions, objects, locations, intentions, quantities). Each gist is prefixed with the episode timestamp where available, and relative time expressions are resolved to absolute dates. Separately, the LLM extracts facts as (subject, predicate, object) triples, each optionally annotated with temporal qualifiers (point_in_time, start_time, end_time). Both are kept even if contradictory, so the memory remains historically revisitable. These become a hybrid memory graph with two node types (gist nodes at the context level, phrase nodes at the concept level) and three edge types: relation edges linking subject and object phrases with a predicate and validity interval; context edges linking a gist to the phrases extracted from the same source chunk; and synonymy edges linking gists whose embeddings are highly similar (threshold 0.8, following HippoRAG 2). Embedding and BM25 lexical indices are maintained over gist text and phrase names.

Phase 2: Agentic inference. Rather than a single similarity match, a ReAct-style agent iterates over the graph with three categories of curated tools. Retrieval tools (semantic_retrieve using an embedding model, lexical_retrieve using BM25) return seed nodes and their contexts, optionally filtered by time range and operators. Graph exploration tools (find_gist_contexts for episode-level narratives, find_entity_contexts for entity-targeted queries with arguments for subject, object, predicate, time filters, limit, ordering, offset, and aggregation) expand from those seeds. Flow control uses output_answer once the agent is confident. Every retrieval or exploration call returns both gists and facts, so the agent always sees memory at both the context and concept level.

Experimental setup. The default LLM is GPT-4.1-mini-2025-04-14 and the default embedding model is nvidia/NV-Embed-v2. Baselines retrieve the top-10 passages or processed chunks; REMem operates over the top-10 gists and facts; HippoRAG 2 uses the top-3 returned sessions. Two variants are tested: REMem-I (iterative, up to a maximum number of agentic steps chosen between 2 and 5 on a small validation set; 3 for recollection tasks and 5 for reasoning tasks) and REMem-S (single-step embedding retrieval followed by generation). Metrics are token-based F1, BLEU-1, and LLM-as-a-judge scores for QA tasks, and exact match for Test of Time. LoCoMo and REALTALK cover episodic recollection; Complex-TR (1,000 randomly sampled queries) and the semantic part of Test of Time cover episodic reasoning.

Why This Matters

Impact on research. The paper reframes agent memory away from semantic-knowledge retrieval and toward explicit event modeling with spatiotemporal grounding, offering a formal definition (a typed multigraph with gist and phrase nodes and relation, context, and synonymy edges) along with an evaluation protocol that separates recollection from reasoning. It provides a reproducible open-source baseline, shows that LLM-as-a-judge agrees with human judgment at 93% on the tested sample, and demonstrates that one-time similarity retrieval is insufficient for multi-step queries over interaction histories.

Real-world applications:

• Personal assistants and chatbots that need to recall what a user did, when, and with whom across many sessions, and to answer questions like "which of my two pets did I adopt more recently?"
• Customer-support agents that must reconstruct a time-ordered history of a customer's interactions, filter by time windows, and refuse to answer when the record contains nothing relevant.
• Long-horizon task agents that must track their own past actions and their ordering to plan subsequent steps.
• Temporal reading comprehension and document analysis over large corpora, where answers depend on date resolution, counting, duration, and ordinal constraints.

Industry relevance. The work is a collaboration between The Ohio State University and Intuit AI Research, with code and data released publicly. Its measured token efficiency (9k input tokens per LoCoMo query for REMem-I versus 26k for Full-Context) and its refusal behavior directly address the cost and reliability concerns that matter when deploying memory-backed agents in production. The authors note that building memories in a streaming format remains an engineering challenge compared with offline batch indexing.

Future Directions

• Extending the framework to long-term memory for language agents operating in more complex environments than the four evaluated benchmarks.
• Engineering streaming memory construction, which the authors flag as an open challenge relative to offline batch indexing.
• Reducing the error modes surfaced in the analysis: selection and grounding errors (46% of sampled LoCoMo errors), temporal window mismatches (42% of sampled Complex-TR errors), incomplete multi-entity lists (about 21%), and offset-direction confusion (about 18%).
• Determining when single-step retrieval suffices versus when iterative agentic retrieval is worth the extra steps, given that REMem-S outperformed REMem-I on several recollection metrics and that cross-session questions account for only 14.2% of LoCoMo.

Target Audience

Researchers and engineers working on LLM agent memory, retrieval-augmented generation, and long-horizon conversational systems; practitioners building personalization, customer-support, or task agents that must reason over interaction histories; and evaluation researchers interested in temporal reasoning benchmarks and in the validity of LLM-as-a-judge scoring for memory tasks. Readers should be comfortable with RAG pipelines, knowledge graphs, and agent tool-use protocols, though the paper's two-phase structure is explained concretely enough for a motivated newcomer.

Authors’ abstract

Humans excel at remembering concrete experiences along spatiotemporal contexts and performing reasoning across those events, i.e., the capacity for episodic memory. In contrast, memory in language agents remains mainly semantic, and current agents are not yet capable of effectively recollecting and reasoning over interaction histories. We identify and formalize the core challenges of episodic recollection and reasoning from this gap, and observe that existing work often overlooks episodicity, lacks explicit event modeling, or overemphasizes simple retrieval rather than complex reasoning. We present REMem, a two-phase framework for constructing and reasoning with episodic memory: 1) Offline indexing, where REMem converts experiences into a hybrid memory graph that flexibly links time-aware gists and facts. 2) Online inference, where REMem employs an agentic retriever with carefully curated tools for iterative retrieval over the memory graph. Comprehensive evaluation across four episodic memory benchmarks shows that REMem substantially outperforms state-of-the-art memory systems such as Mem0 and HippoRAG 2, showing 3.4% and 13.4% absolute improvements on episodic recollection and reasoning tasks, respectively. Moreover, REMem also demonstrates more robust refusal behavior for unanswerable questions.