Beyond Fact Retrieval: Episodic Memory for RAG with Generative Semantic Workspaces

AI summary

Overview

Research area: long-context reasoning, retrieval-augmented generation (RAG), and memory architectures for large language models. Technical level: Intermediate. Scope: This paper introduces the Generative Semantic Workspace (GSW), a two-module memory framework that builds structured, spatiotemporally grounded representations of actors, roles, states, and interactions from long narratives, and evaluates it against RAG baselines on the Episodic Memory Benchmark (EpBench).

What This Paper Is About

Large language models handle long documents poorly: many corpora exceed their context windows, and accuracy degrades with sequence length and when relevant information sits in the middle of a context. Existing fixes, from dense embedding retrieval to structured knowledge graphs, are built for fact lookup and do not track entities through evolving events across space and time. The goal of this paper is to give LLMs an episodic memory: a persistent, interpretable workspace that records who did what, where, when, how roles changed, and what consequences followed, and that answers queries by returning short, entity-specific summaries instead of whole document chunks.

Key Contributions

1. The GSW framework: a neuro-inspired generative memory model with two components — an Operator that maps each incoming text observation into an intermediate semantic structure, and a Reconciler that integrates these into a persistent workspace enforcing temporal, spatial, and logical coherence.
2. A formal probabilistic account of the workspace as a sampled distribution over actors, roles, states, verbs, time, space, and forward-looking questions, with a recursive state-space update rule (Markovian transition model) for reconciling new context into the prior workspace.
3. An end-to-end question-answering procedure over the workspace: match query entities to workspace nodes, generate contextual episodic summaries for matched entities, re-rank them by semantic similarity to the query, and pass only the top-ranked summaries to the LLM.
4. An empirical evaluation on two scales of EpBench (200-chapter and 2000-chapter corpora) showing state-of-the-art accuracy alongside large reductions in query-time token usage, with code released publicly.

Main Findings

• Strongest overall accuracy on EpBench-200 (686 QA pairs): GSW reaches an overall F1 of 0.850, precision 0.865, and recall 0.894. The next-best overall F1 is Embedding RAG at 0.771, followed by HippoRAG2 at 0.753, GraphRAG at 0.714, LightRAG at 0.678, and Vanilla LLM at 0.629 (the discussion text cites 0.642 for the Vanilla LLM, which differs slightly from the tabulated value). GSW scores highest in 16 of 18 individual metric computations and second in the remaining two.
• The hardest category is where the gap widens: for queries with 6+ matching cues, where answers can require reasoning across up to 17 distinct chapters, GSW records F1 0.834, precision 0.890, and recall 0.822 — roughly a 20% recall improvement over HippoRAG2, the next most performant method in that category (recall 0.675). The paper notes that recall falls as cue count rises for all competing frameworks, while GSW stays consistent.
• Best accuracy at low-cue and cue-free queries: GSW scores 0.978 on precision, recall, and F1 for the 0-cue category, which the paper associates with reduced hallucination when no matching cues appear in the source document.
• Substantial token efficiency: GSW uses about 3,587 average context tokens per query, versus about 7,340 for GraphRAG, about 8,771 for Embedding RAG and HippoRAG2, about 40,476 for LightRAG, and about 101,120 for the Vanilla LLM. This is a 51% reduction against the next most token-efficient baseline (GraphRAG) and nearly 59% against Embedding RAG and HippoRAG2. Estimated cost per query on GPT-4o pricing ($2.50 per million tokens) is about $0.0090 for GSW versus about $0.0184 for GraphRAG and about $0.2528 for the Vanilla LLM.
• Performance persists at 10x scale: on EpBench-2000, GSW achieves overall F1 0.773, precision 0.830, and recall 0.796. The paper reports this as 15% higher than the strongest baseline (Embedding RAG, F1 0.675) and 22% higher than other structured RAG methods; the abstract phrases the gain as more than 15% on overall recall.
• EpBench-200 corpus scale: 200 chapters, 102,870 total tokens, 686 QA pairs, distributed as 180 / 180 / 108 / 128 / 90 queries over the 0, 1, 2, 3–5, and 6+ cue categories, with a minimum of 0 and a maximum of 17 chapters referenced per query.

Methodology in Plain English

The authors start from an analogy to how brains are thought to handle memory: cortical regions store abstractions of entities, roles, and event templates, while the hippocampal system binds them into coherent sequences in space and time, with replay consolidating them. GSW splits the work accordingly.

The Operator reads text in short, semantically coherent chunks and extracts structured semantics — actors, their roles, their states, the verbs describing interactions, spatiotemporal coordinates, and open questions about likely outcomes. The Reconciler then merges each new extracted structure into a single global memory, checking that times, places, and logic stay consistent and updating prior entries as new evidence arrives. Because updates are recursive and Markovian, the workspace grows incrementally as documents stream in rather than being rebuilt.

To answer a question, the system matches named entities in the query to nodes in the workspace using string matching, assembles episodic summaries around those entities, re-ranks the summaries by similarity to the query, and passes only the top ones to the LLM. The implementation prompts GPT-4o with task-specific instructions at temperature 0 for the Operator and Reconciler, and GPT-4o also generates final answers for all compared methods. Context was capped at 17 chapters per query for every method to keep the comparison fair. Scoring uses an LLM-as-a-Judge paradigm following the EpBench authors' answer extraction procedure, with error bars from bootstrap resampling, and baselines are Vanilla LLM, Embedding RAG (Voyage-03 embeddings), GraphRAG, HippoRAG2, and LightRAG.

Why This Matters

The paper argues that most text LLMs encounter is not fact lists but narrative — crime reports, political briefings, corporate filings, legislative records, war dispatches, multi-day news — where the same actors change roles and states over time. Indexing such documents by a world model rather than by chunk similarity changes what question answering can do, and it does so while cutting inference cost, since fewer tokens are sent per query.

Real-world applications:

• Investigative and legal work: tracking a suspect through arrest, arraignment, and release across a large case file, or following a regulator's actions over time.
• Corporate and financial analysis: following a company from startup to unicorn to acquisition, and tracking bidders, filings, and roles across years of disclosure documents.
• Journalism and open-source intelligence: maintaining a coherent record of multi-day or multi-week coverage where the same locations, dates, and people recur.
• Long-horizon agent memory: giving deployed agents a persistent, interpretable store of past situations they can query cheaply.

Industry relevance: the framework is described as plug-and-play, requiring no specialized training or fine-tuning and working with any LLM, which makes it a drop-in memory layer for existing RAG pipelines. The token and cost reductions — about $0.0090 versus about $0.0184 per query against the next most efficient baseline in the paper's setup — speak directly to serving costs at scale.

Future Directions

• Stronger episodic benchmarks: the authors say EpBench's scope limits how thoroughly it probes complex evolution of actor roles and states within extended narratives, and they are developing a more comprehensive benchmark.
• Model independence: the current implementation depends on a strong closed-source LLM (GPT-4o), and empirically validating open-source alternatives inside the framework is identified as essential.
• Beyond text: expanding GSW to data modalities other than text is named as an important direction.
• Component-level ablation and retrieval variants: the full version reports ablations removing identifier types such as temporal and spatial tags, evaluations on a shortened EpBench version, and comparisons across different retrieval strategies.

Target Audience

Researchers and engineers working on long-context reasoning, retrieval-augmented generation, structured knowledge representations, and agent memory will benefit most. It is also relevant to practitioners who need to answer compositional questions over large narrative corpora at controlled inference cost, and to readers interested in biologically motivated computational models of episodic memory. The paper's formal sections use probabilistic notation, but the framework and results are accessible without deep mathematical background.

Authors’ abstract

Large Language Models (LLMs) face fundamental challenges in long-context reasoning: many documents exceed their finite context windows, while performance on texts that do fit degrades with sequence length, necessitating their augmentation with external memory frameworks. Current solutions, which have evolved from retrieval using semantic embeddings to more sophisticated structured knowledge graphs representations for improved sense-making and associativity, are tailored for fact-based retrieval and fail to build the space-time-anchored narrative representations required for tracking entities through episodic events. To bridge this gap, we propose the \textbf{Generative Semantic Workspace} (GSW), a neuro-inspired generative memory framework that builds structured, interpretable representations of evolving situations, enabling LLMs to reason over evolving roles, actions, and spatiotemporal contexts. Our framework comprises an \textit{Operator}, which maps incoming observations to intermediate semantic structures, and a \textit{Reconciler}, which integrates these into a persistent workspace that enforces temporal, spatial, and logical coherence. On the Episodic Memory Benchmark (EpBench) \cite{huet_episodic_2025} comprising corpora ranging from 100k to 1M tokens in length, GSW outperforms existing RAG based baselines by up to \textbf{20\%}. Furthermore, GSW is highly efficient, reducing query-time context tokens by \textbf{51\%} compared to the next most token-efficient baseline, reducing inference time costs considerably. More broadly, GSW offers a concrete blueprint for endowing LLMs with human-like episodic memory, paving the way for more capable agents that can reason over long horizons. Code is available at https://github.com/roychowdhuryresearch/gsw-memory.