MemWeaver: A Hierarchical Memory from Textual Interactive Behaviors for Personalized Generation

AI summary

Overview

• Research area: Natural Language Processing / personalized text generation, memory-augmented LLMs, user modeling from textual interaction histories.
• Technical level: Intermediate (requires familiarity with retrieval-augmented generation, LLM prompting, and evaluation metrics such as ROUGE and MAE).
• Scope: The paper introduces MemWeaver, a framework that converts a user's chronological textual history into a two-part hierarchical memory — a behavioral memory extracted from a graph and a cognitive memory abstracted through hierarchical summarization — and evaluates it on six datasets from the LaMP benchmark.

What This Paper Is About

Personalization systems have traditionally relied on implicit signals such as clicks and purchases, representing users as vectors over item IDs. As user-internet engagement shifts toward explicit textual interactions, a rich written history becomes available — but prevailing methods treat that history as a flat list of texts to retrieve from, ignoring the temporal evolution of interests and the semantic connections between activities. MemWeaver's goal is to weave that history into a structured hierarchical memory that lets an LLM reason over both concrete past behaviors and abstract long-term preferences.

Key Contributions

1. The MemWeaver framework, which weaves user history into a hierarchical memory, moving beyond the representational limits of conventional flat logs.
2. A memory construction methodology with two components: an extracted behavioral memory that captures concrete, query-specific context, and an abstracted cognitive memory that represents evolving long-term preferences.
3. An incremental update mechanism for both memory components, designed to handle continuously growing user histories without full reconstruction.
4. Extensive experiments on the LaMP benchmark (six datasets) reporting state-of-the-art performance and an in-depth analysis confirming the synergy of the dual-memory components.

Main Findings

• Top performance on all twelve evaluation metrics: MemWeaver ranked first on all twelve metrics across the six LaMP datasets, using both Qwen3-8B and Llama-3.1-8B-Instruct as backbones. The paper reports all improvements over baselines as statistically significant (t-tests, p-value < 0.05).
• Representative scores with Qwen3-8B: LaMP-1 Acc. 0.6733 / F1 0.3367; LaMP-2 Acc. 0.4633 / F1 0.0311; LaMP-3 MAE 0.2800 / RMSE 0.3733; LaMP-4 R-1 0.1724 / R-L 0.1544; LaMP-5 R-1 0.4757 / R-L 0.4138; LaMP-7 R-1 0.4792 / R-L 0.4079.
• Representative scores with Llama-3.1-8B-Instruct: LaMP-1 Acc. 0.6533 / F1 0.3267; LaMP-2 Acc. 0.4733 / F1 0.0315; LaMP-3 MAE 0.2533 / RMSE 0.3300; LaMP-4 R-1 0.1899 / R-L 0.1794; LaMP-5 R-1 0.4718 / R-L 0.4138; LaMP-7 R-1 0.3581 / R-L 0.2916.
• Both memories are indispensable, but hierarchically related: Removing behavioral memory caused a catastrophic collapse on context-heavy generation, with LaMP-4 R-1 dropping to 0.1069 and LaMP-3 MAE rising to 0.6400 and RMSE to 1.2500. Removing cognitive memory produced a comparatively smaller drop, supporting the view of behavioral memory as the foundational context layer and cognitive memory as a refinement layer.
• Edge weighting is the most critical behavioral component: The variant without edge weighting suffered the most severe degradation (LaMP-1 Acc. 0.5708, F1 0.2854), showing that building a behavior graph is not enough — quantifying the relationships matters.
• Semantic edges matter more than temporal edges for accuracy, though temporal edges are traversed more: Dropping semantic edges hurt performance more than dropping temporal edges, yet temporal edges accounted for the majority of traversals across all datasets. The paper interprets temporal edges as the high-frequency backbone and semantic edges as bridges connecting conceptually related but temporally distant activities.
• Hierarchical summarization beats one-shot summarization: Removing the clustering/segmentation step degraded performance, and removing the global summary caused a consistent drop, indicating that segmenting history into phases and then integrating them captures both distinct interests per period and their evolution.
• Incremental updates are efficient: Benchmarked against Full Rebuild and No Update, the incremental strategy closely tracked Full Rebuild accuracy while keeping computational overhead minimal, comparable to the near-instantaneous No Update baseline.
• Hyperparameters need task-specific tuning: Moderate values of the semantic guidance weight α worked best; both negligible and overly strong guidance were suboptimal. Optimal recency (λ1) and continuity (λ2) biases were task-dependent — a moderate preference for recent or contiguous interactions helped in scholarly title generation (LaMP-5), while strong bias was overly restrictive for tweet paraphrasing (LaMP-7), where temporal coherence had a relatively small effect.
• Case study: For a user with interests in vintage jazz, specialty coffee brewing, and DIY crafts who states that a great gift is about the experience, not the object, the memory-less LLM suggested generic items (a food basket or a book), while MemWeaver recommended a siphon brewing kit aligned with the user's stated belief.

Methodology in Plain English

The framework starts with a user's chronologically ordered history of documents. It builds two memories from that history.

For the behavioral memory, each document is encoded into a dense vector using a pre-trained sentence-level transformer (BGE-M3). A K-means clustering step partitions the user's behaviors into K semantic clusters representing thematic interests. These clusters plus chronology define a personalized graph: each behavior is a node; temporal edges link consecutive behaviors, and semantic edges link any two nodes inside the same cluster. To pull out the part of the graph relevant to the current query, the framework runs a context-aware random walk — inspired by how one memory triggers related recollections. The walk's transition score multiplies a semantic relevance term (the query-to-node cosine similarity raised to a power α) by a recency score and a continuity score, both exponential decays controlled by λ1 and λ2. Nodes visited up to a step limit form the behavioral memory.

For the cognitive memory, the history is split into temporally coherent segments at semantic breakpoints — points where similarity between consecutive behaviors drops sharply — with rule-based constraints on segment size. An LLM summarizes each segment into a local summary, and then a second pass acts as a preference integrator, weaving the local summaries into one global, narrative-style profile.

Generation then conditions the LLM jointly on the query, the behavioral memory, and the cognitive memory. Behavioral memory supplies concrete short-term grounding; cognitive memory supplies high-level long-term guidance. Updates are incremental: new behaviors are encoded and attached to the graph by linking the last existing node to the first new one and clustering only the new batch, while the cognitive memory is re-synthesized from the full collection of local summaries.

Implementation used Qwen3-8B and Llama-3.1-8B-Instruct as backbones and BGE-M3 as the retrieval model. Experiments ran on a server with 2 Nvidia GeForce RTX 4090 GPUs (24GB memory each).

Why This Matters

This work argues that personalization should move from ID-level signals to interpreting the semantics of users' own words, and it shows a concrete structure — a graph-derived behavioral memory plus an abstracted cognitive profile — that outperforms flat retrieval on a standard benchmark. It also demonstrates that semantic and temporal signals must be integrated jointly rather than traded off, and that memory can be maintained incrementally for streaming settings.

Real-world applications:

• Personalized content and recommendation assistants that adapt suggestions to a user's evolving tastes rather than a static profile.
• Consumer chat and shopping agents that give gift or product recommendations grounded in a user's stated values and past interactions.
• Writing assistants for headlines, titles, and paraphrasing that match an individual's stylistic and topical history (the tasks covered by LaMP-4, LaMP-5, LaMP-7).
• Streaming/continuous personalization services where histories grow constantly and full recomputation is too costly, aided by the incremental update mechanism.

Industry relevance: the incremental update design targets deployment latency and token cost, which matters for production systems serving many users with growing textual histories. The approach is model-agnostic across the two backbones tested.

Future Directions

• Adaptive hyperparameter selection: Since optimal recency (λ1), continuity (λ2), and semantic guidance (α) settings were task-dependent, automatically setting them per task or per user is an open problem.
• Determining the right granularity: The number of semantic clusters K and the segment-size rules are design choices; the paper's ablation shows segmentation and clustering matter but does not establish an optimal configuration.
• Broader dataset validation: LaMP-6 was excluded because it is not publicly available, and the paper notes additional experiments on other datasets are detailed in Appendix C, which is outside the provided content. Extending validation beyond LaMP remains a natural step.
• Longitudinal and dynamic evaluation: The incremental update study simulates sequential arrival of interaction data; how well memory quality and personalization hold up over long deployment horizons, including with noisy or contradictory new behaviors, is not fully resolved.

Target Audience

Researchers and practitioners working on personalized text generation, retrieval-augmented generation, and memory-augmented LLMs. It will be most useful to readers who already understand retrieval pipelines and LLM prompting and want to see how graph structure and hierarchical summarization can replace flat, single-dimensional retrieval. Recommender-systems researchers interested in the shift from implicit ID-level feedback to explicit textual interaction histories will also find the framing relevant. Beginners can follow the high-level design and the case study, but the transition-scoring formula and ablation tables benefit from an intermediate background.

Authors’ abstract

The primary form of user-internet engagement is shifting from leveraging implicit feedback signals, such as browsing and clicks, to harnessing the rich explicit feedback provided by textual interactive behaviors. This shift unlocks a rich source of user textual history, presenting a profound opportunity for a deeper form of personalization. However, prevailing approaches offer only a shallow form of personalization, as they treat user history as a flat list of texts for retrieval and fail to model the rich temporal and semantic structures reflecting dynamic nature of user interests. In this work, we propose \textbf{MemWeaver}, a framework that weaves the user's entire textual history into a hierarchical memory to power deeply personalized generation. The core innovation of our memory lies in its ability to capture both the temporal evolution of interests and the semantic relationships between different activities. To achieve this, MemWeaver builds two complementary memory components that both integrate temporal and semantic information, but at different levels of abstraction: behavioral memory, which captures specific user actions, and cognitive memory, which represents long-term preferences. This dual-component memory serves as a comprehensive representation of the user, allowing large language models (LLMs) to reason over both concrete behaviors and abstracted cognitive traits. This leads to content generation that is deeply aligned with their latent preferences. Experiments on the six datasets of the Language Model Personalization (LaMP) benchmark validate the efficacy of MemWeaver. Our code is available.