Jev-Mem: System-One-Controlled Agentic Memory for Efficient AI Agents

AI summary

Overview

• Research area: Agentic memory for large language model (LLM) agents, sitting at the intersection of long-horizon agent memory systems and cost-aware LLM inference.
• Technical level: Advanced. The paper contains formal notation for memory structures, typed probabilistic control decisions, retrieval budget allocation, and scoring functions, plus a systems-level efficiency evaluation.
• Scope: The paper proposes and evaluates Jev-Mem, a three-plane agentic memory architecture (System-One control plane, structured multi-relational memory plane, System-Two reasoning plane) that is compared against five baselines on the LoCoMo benchmark for answer quality, memory build time, and query latency.

What This Paper Is About

Persistent AI agents accumulate far more user preferences, task history, and environment knowledge than fits in a fixed context window, so they need external "agentic memory" that can store, organize, and retrieve past experience. The problem the paper targets is that existing memory systems control these memory operations either with inflexible fixed heuristics or by repeatedly calling autoregressive LLMs to make high-frequency, bounded decisions (what type is this memory, what is it related to, where should I search, when should I stop) — placing expensive token generation on the critical path. Jev-Mem's goal is to move those structured, high-frequency memory-control decisions into a lightweight "System-One" controller while reserving the expensive LLM ("System Two") only for complex reasoning and answer synthesis, improving both accuracy and runtime efficiency.

Key Contributions

1. The paper identifies an opportunity to use System-One-style lightweight decision-making (fast, automatic, structured) for agentic memory, arguing that many memory-control operations are semantic but not generative, producing only labels, probabilities, or scores.
2. It introduces Jev-Mem, a System-One/System-Two architecture with three components: a System-One controller, a shared memory data plane, and a System-Two reasoning model. Memory control is treated as a first-class systems layer rather than as prompts or heuristics buried inside the memory pipeline.
3. It develops a unified System-One control plane used across both the write path (memory typing, redundancy filtering, semantic/temporal/causal/entity relation construction) and the read path (query routing, retrieval-budget allocation, graph traversal, candidate scoring, evidence assessment, adaptive stopping).
4. It reports that Jev-Mem achieves the best overall accuracy while also delivering the highest efficiency among the evaluated state-of-the-art baselines on LoCoMo, and the code is publicly available at the linked GitHub repository.

Main Findings

• Overall LoCoMo accuracy: Jev-Mem reaches an overall LLM-as-a-Judge score of 0.777, versus 0.700 for the strongest baseline (MAGMA), described as an 11.0% relative improvement. The judge LLM is gpt-4o-mini.
• Per-category scores (Table 1): Jev-Mem reports Multi-Hop 0.623, Temporal 0.637, Open-Domain 0.618, Single-Hop 0.802, Adversarial 0.962, Overall 0.777. Baselines in the same table: Full Context (0.468 / 0.562 / 0.486 / 0.630 / 0.205 / 0.481), A-MEM (0.495 / 0.474 / 0.385 / 0.653 / 0.616 / 0.580), MemoryOS (0.552 / 0.422 / 0.504 / 0.674 / 0.428 / 0.553), Nemori (0.569 / 0.649 / 0.485 / 0.764 / 0.325 / 0.590), MAGMA (0.528 / 0.650 / 0.517 / 0.776 / 0.742 / 0.700).
• Text–table discrepancies in the paper: the narrative text states Multi-Hop 0.625, Open-Domain 0.610, Single-Hop 0.797, and that Jev-Mem "matches the best Temporal score of 0.650," while Table 1 lists 0.623, 0.618, 0.802, and 0.637 respectively. The paper also says Jev-Mem performs best in five of the six categories and matches the best result on temporal reasoning.
• Largest gains: the largest improvement is on Adversarial questions (0.962 vs 0.742 for the strongest baseline), with clear gains on Multi-Hop and Open-Domain, which the paper attributes to adaptive memory control selecting a more relevant and focused evidence set.
• Memory construction speed: Jev-Mem builds memory in 158 s, versus 1,044 s for the fastest competing memory system (Nemori), reported as an 84.9% reduction or a 6.6× speedup. A-MEM requires 3,636 s and MemoryOS 3,276 s; MAGMA is listed at 1,404 s; Full Context is listed as N/A.
• Query latency: Jev-Mem averages 0.93 s per query, 36.7% lower than the fastest memory-based baseline at 1.47 s (MAGMA) and 46.6% lower than processing the full context at 1.74 s. A-MEM is 2.26 s, Nemori 2.59 s, and MemoryOS 32.68 s.
• Efficiency without an accuracy trade-off: the paper emphasizes that the efficiency gains come alongside the highest overall accuracy, and attributes them to batched typed decisions over a bounded candidate set, budget-constrained graph exploration, and adaptive stopping.
• Benchmarks covered: the setup section states evaluation is on "two widely used benchmarks" for long-term conversational memory, but only LoCoMo is named and reported in the provided content; the second benchmark is not named there, and no second-benchmark results are reported. The paper also reports no ablation studies, hyperparameter sensitivity analysis, or per-decision accuracy measurements in the provided content.

Methodology in Plain English

The researchers start from a diagnosis rather than a new memory representation: memory systems spend a lot of LLM compute on small, repetitive decisions. They separate those decisions from the ones that genuinely need deliberation.

1. Represent memory once, relate it many ways. Each observation becomes a canonical memory node holding its content, optional timestamp, provenance, embedding, entities, and type scores. Rather than filing a memory into one category, the system scores it on four overlapping characteristics — episodic, semantic, procedural, and preference — so one memory can be several things at once. Edges of four types (semantic, temporal, causal, entity) can coexist between the same pair of nodes, all over the same shared node set, with vector and lexical indexes providing entry points.
2. Replace LLM prompting with typed questions. A "typed System-One controller" takes a structured state and a batch of explicit questions with known, small answer spaces. It returns probabilities for independent propositions (are these two memories semantically related? is this candidate relevant?) or a distribution over mutually exclusive alternatives (for example, temporal relations among before, after, during, contains, overlaps, same_time, and unknown). Because outputs are bounded, they can be batched and read directly instead of generating and parsing text.
3. Control the write path. Instead of comparing each new observation to every existing memory, the system first uses deterministic signals (vector similarity, lexical overlap, shared entities, temporal proximity) to propose at most K_w candidates, then asks the controller about only those pairs. Edges are inserted only above a relation threshold. Timestamps and exact shared identifiers create temporal and entity edges directly, avoiding learned inference when structured information is already available.
4. Control the read path as a closed loop. For each query, the controller predicts which relational views are relevant, whether multi-hop reasoning is needed, and how important recency is. Budget is then split across active views, and traversal depth scales with the multi-hop prediction. Entry points come from fusing vector and keyword rankings with reciprocal-rank fusion. After each round, the controller estimates evidence sufficiency, the expected utility of further retrieval, missing required evidence, and unresolved contradiction, and stops when evidence is sufficient or further search looks unlikely to help. Candidates are scored on relevance, relation usefulness, novelty, and support for current evidence, combined with embedding similarity and edge weight, with a recency adjustment when timestamps exist. The top W candidates form the next beam.
5. Send only the final step to the big model. After retrieval terminates, the highest-scoring K memories go to the System-Two LLM for synthesis. The paper stresses that System Two is not involved in routing, expansion, or stopping, and that control overhead is explicitly bounded by limits on write requests, retrieval rounds, expansions, inspected edges, visited nodes, depth, controller invocations, and elapsed time.
6. Evaluate on accuracy and cost together. The evaluation reports the LLM-as-a-Judge score, total memory construction time, and average per-query latency on LoCoMo against Full Context, A-MEM, Nemori, MemoryOS, and MAGMA, using the same backbone answer model whenever applicable.

Why This Matters

The paper reframes memory management as a systems resource-allocation problem instead of a prompting problem. For research, it suggests that the cost of memory control — not just what is stored or how retrieval is structured — is a design axis with measurable consequences, and it proposes a reusable abstraction (typed, batched, bounded decisions) that could be swapped for other System-One realizations. It also provides matched accuracy-and-efficiency numbers where many memory papers report only accuracy.

Real-world applications the paper frames this around:

• Coding assistants that must retain repository and task context across long sessions.
• Personal agents that accumulate user preferences and interaction history over time.
• Research agents that build up environment and domain knowledge across many steps.
• Autonomous workflows operating over multi-step environments such as interactive web platforms and software repositories.

Industry relevance: lower per-query latency (0.93 s reported) and much lower memory build time (158 s reported) translate directly into serving cost and user-perceived responsiveness for persistent agent products, since memory operations occur on every interaction rather than once. The paper also notes that when memory management remains on the retrieval path, overhead can become severe — MemoryOS is reported at 32.68 s per query, which is impractical for interactive deployment.

Future Directions

• Calibration and reliability of the System-One controller. The appendix states that the model-reported values in [0, 1] "are not assumed to be calibrated probabilities," which leaves open how thresholds should be chosen and how sensitive results are to that calibration.
• Generalization beyond LoCoMo. The setup mentions two benchmarks but reports only LoCoMo; related work cites LongMemEval, MemBench, and MemoryAgentBench as long-horizon memory benchmarks, and none of these are evaluated here.
• Ablations and component attribution. The paper reports no ablations isolating the contribution of typing, candidate discovery, query routing, budget allocation, candidate scoring, or adaptive stopping, so it is not established which System-One decisions drive the reported gains.
• Other System-One realizations and other memory content. The paper states Jev is only one concrete realization of the System-One controller, and its related-work discussion covers procedural memory, skill libraries, and distilled experience that Jev-Mem's described design does not directly address.

Target Audience

Researchers and engineers working on LLM agent memory, long-context agents, and inference-cost optimization will benefit most. It is also relevant to systems researchers interested in where non-autoregressive, structured prediction can replace generative calls inside an agent pipeline, and to practitioners building persistent assistants who need to weigh answer quality against memory build and query latency. Readers should be comfortable with graph structures, probability thresholds, and retrieval terminology; the architectural argument is accessible, but the design and formulas are not beginner material.

Authors’ abstract

Agentic memory is becoming essential for long-horizon AI agents, yet many existing systems rely on autoregressive LLMs to control how memories are organized, retrieved, and used, placing expensive generation on the critical path of memory operations. We introduce \textbf{\method}, a new agentic memory architecture inspired by System-One/System-Two cognition. System One captures fast, lightweight decision-making, whereas System Two performs slower, deliberative reasoning. Jev-Mem brings this division of labor to agentic memory through a dedicated System-One control plane, a structured multi-relational memory plane, and a System-Two reasoning plane. The System-One controller governs memory typing and relational organization during construction, and dynamically performs query routing, retrieval-budget allocation, graph traversal, candidate scoring, and adaptive stopping during retrieval. System Two is invoked only for complex reasoning and answer synthesis. This design improves both memory effectiveness and system efficiency: on LoCoMo Jev-Mem achieves an overall LLM-as-a-Judge score of 0.777, an 11.0\% relative improvement over the strongest baseline, while reducing memory construction time to 158\,s, a 6.6$\times$ speedup over the fastest competing memory system, and lowering average query latency to 0.93\,s, a 36.7\% reduction.