EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory

AI summary

Overview

Research area: Computer vision / multimodal agent systems — specifically external memory for multi-agent systems (MAS) built on large language and vision-language models.

Technical level: Advanced. The paper assumes familiarity with multi-agent orchestration, retrieval-augmented memory, multimodal (text + image-region) representations, and supervised fine-tuning of small policy models.

Scope in one sentence: The paper introduces EpiCon, a shared multimodal memory framework in which a trained 2B memory controller co-evolves textual guidance and visual evidence within a question, while a trained 2B tree self-organizer consolidates lessons into a persistent experience bank that different agent harnesses and backbones can reuse and expand without updating host model parameters.

What This Paper Is About

Agents can improve by learning from their own past executions, but their experience is usually trapped inside the specific system that produced it, so different agents cannot reuse or build on one another's work. EpiCon attacks this by maintaining a shared, persistent bank of multimodal experience — textual lessons plus the image regions that support them — that sits outside the host model and can be read from and written to by more than one agent system. The goal is collective learning: a system should be able to benefit from experience it did not generate, contribute new experience back, and improve subsequent solving while every host model's parameters stay frozen.

Key Contributions

1. Collective agent learning through shared multimodal memory. EpiCon lets systems share experience without updating host parameters. A frozen bank improves performance across harnesses and backbones with a single solving attempt, and contributions from another harness raise the original system's macro-average score by 2.1–3.2 points.

2. Co-evolution of textual guidance and visual evidence. A memory controller jointly refines guidance text and supporting image regions across attempts, and selectively decides whether visual memory is injected into the next attempt.

3. Hierarchical consolidation and retrieval. A tree self-organizer consolidates lessons hierarchically, abstracts reusable rules at internal nodes, and retrieves applicable experience or rules for new problems — or nothing, when no candidate applies.

4. Efficient memory management with compact trained models. Both modules are independently trained 2B models. Across eleven benchmarks and four host configurations, the 2B variant improves macro-average scores by 1.7–4.9 points over No Memory and reduces memory-operation time by 67–74% relative to backbone-sized memory models.

Main Findings

• Backbone-sized EpiCon leads on macro-average across all four host configurations. With a single solving attempt per question, it achieves the highest eleven-benchmark macro-average in all four host configurations, exceeding the strongest external memory baseline in each by 1.9 to 5.9 points.

• Large per-benchmark gains on specific tasks. For Codex with Qwen3.8-27B, backbone-sized EpiCon improves ParseBench from 57.1 to 68.5 and MATH-Vision from 20.4 to 48.6 over No Memory. Averaged over the two Qwen3.8-27B host configurations, it scores highest on ten of eleven benchmarks.

• Compact 2B memory models trade a small score drop for large speed gains. Replacing backbone-sized memory models with the two trained 2B models reduces memory-operation time by approximately 67% to 74% and total (MAS + memory) time by 25% to 35%, with macro-average scores lower by 0.9 to 3.6 points. The 2B variant still improves macro-average scores over No Memory by 1.7 to 4.9 points across the four host configurations.

• Gains transfer to task types the memory models were not trained on. Training supervision came from visual mathematics, document understanding, and chart/infographic reasoning. The 2B variant improves WorldBench in all four configurations (+3.2 to +7.8 points), while changes on ReasonMap (+0.5 to +1.9) and BabyVision (−2.7 to +4.8) are smaller and less consistent.

• A bank transfers across backbones and harnesses. A bank built by Codex with Qwen3.8-27B, reused by Codex with Gemma4-31B or by DeepSeek-Harness with Qwen3.8-27B, yields macro-average improvements of 3.2 and 4.2 points respectively. Backbone transfer improves MATH-Vision from 25.8 to 33.0; harness transfer improves ParseBench from 52.4 to 65.7.

• Cross-harness evolution helps the original bank builder. Macro-average scores rise from 53.5 to 55.7 for Codex and from 53.4 to 56.6 for DeepSeek-Harness — gains of 2.1 and 3.2 points. The evolved banks improve eight and nine of eleven benchmark scores, respectively, evaluated on the same 4,388 questions.

• Transfer benefits are task-dependent. On ParseBench, Codex's score drops from 63.4 to 58.9 after DeepSeek-Harness evolves its bank, while DeepSeek-Harness improves from 52.3 to 65.9 after Codex evolves its bank. On MATH-Vision, Codex changes only slightly from 45.2 to 45.4, while DeepSeek-Harness declines from 56.6 to 53.4.

• Question-level memory control works with or without a bank. With the trained 2B controller and no historical bank, refinement across up to five attempts improves all four measured scores under both harnesses, including Vision2Code from 53.7 to 57.8 with DeepSeek-Harness. The larger Qwen3.8-27B controller scores higher in seven of eight comparisons, while the 2B controller requires less MAS execution time and no more memory-operation time.

• Visual memory helps, and adaptive injection helps more. With always-on injection, frozen visual memory improves Codex's MATH-Vision score from 27.2 to 40.6 over text-only memory. Evolving visual memory improves seven scores over frozen memory and ties the remaining score. Adaptive injection improves seven of eight scores with either frozen or evolving visual memory and reduces MAS execution time by approximately 25% with Codex and 17% with DeepSeek-Harness.

• Hierarchical tree organization beats a flat bank. Using lessons derived from the same source questions, tree memory improves all scores over flat memory, including ParseBench from 52.2 to 67.3 and BabyVision from 13.2 to 15.3.

• EpiCon still helps a stronger API-based solver. On Codex with GPT-5.6-Luna, with the 2B components and historical bank fixed and no target-specific retraining, EpiCon improves all four benchmarks: ParseBench 63.8 to 66.5, Vision2Code 65.8 to 67.5, MATH-Vision 51.2 to 57.2, and BabyVision 11.1 to 14.6.

Methodology in Plain English

EpiCon has three moving parts.

A memory harness coordinates everything: it retrieves experience, triggers memory updates, validates proposed changes, performs visual crops, and writes accepted lessons into the bank. The host multi-agent system keeps its own orchestration and backbone untouched.

A Memory Controller (a trained 2B model) manages memory within a question. Memory is a pair: text describing actionable guidance, applicability conditions, and cautions, plus a pointer to a supporting region in a source image. Given the question, images, the previous memory, the latest answer, and the execution trace, the controller proposes a revised text, a source image, and a region — it can keep, replace, or drop the visual evidence as the guidance evolves. It also outputs a gate deciding whether visual memory is injected into the next attempt; the gate controls what is fed in, not what is stored. Proposals are validated by the harness, and invalid ones leave memory unchanged. This loop can run for up to five attempts.

A Tree Self-Organizer (a second, independently trained 2B model) manages memory across questions. The shared bank is a tree: leaves hold question-specific lessons, internal nodes group and summarize related experience, and qualified summaries can become reusable rules. The organizer proposes Place, Merge, Split + Lift, and Consolidation operations; the harness validates and executes them and enforces capacity-based partitions. For a new question, retrieval first recalls candidates from a frozen text encoder over node indices, then the organizer selects applicable rules, a concrete experience, or nothing.

Training data. Roughly 25K controller supervision examples and roughly 6.5K tree-operation demonstrations were built from MAS trajectories run with CAMEL, Codex, and DeepSeek-Harness on tasks from MathNet, MathV360K, ChartQA, InfoVQA, DocVQA, and ChartNet. Qwen3.8-27B acted as the initial teacher proposing updates and operations; Qwen3.8-Flash-Next selectively revised unsuccessful updates and corrected tree-operation targets. Crucially, controller targets were filtered by replay: a proposed memory update was only kept if running it with the same MAS host turned a failed attempt into a successful one (Repair), or preserved success while shortening the text without increasing crop area (Compress). Tree operations were checked for schema validity, node references, partition coverage, selection constraints, and provenance, but not individually verified through downstream replay. The two memory policies were fine-tuned independently from Qwen3.5-2B.

Evaluation. Eleven benchmarks covering 4,538 questions span four domains: document understanding, visual-to-code generation, vision-grounded mathematics, and general visual-language reasoning. Two MAS harnesses (Codex, DeepSeek-Harness) were paired with Qwen3.8-27B and Gemma4-31B, plus Codex with GPT-5.6-Luna. External memory baselines (Mem0, Cognee, A-Mem, Agent-KB) used the corresponding backbone zero-shot and built their own banks from a shared set of 1,000 construction questions disjoint from evaluation questions. All historical-memory evaluations use a single solving attempt per question with question-level updates disabled, and banks stay frozen. Experiments ran on NVIDIA H100 GPUs; GPT-5.6-Luna was accessed through its API.

Why This Matters

Impact on research. The paper reframes agent memory as a shared substrate rather than a per-agent accessory. Two design claims are distinctive: (a) multimodal memory co-evolution is tested separately from historical retrieval, so the benefit of reusing a bank is measured with a single solving attempt and cannot be confused with extra retries; and (b) the "collective learning" claim is tested bidirectionally — a system uses another's bank, then contributes experience that improves the original contributor. The 2B design point also shows that memory policies can be decoupled and distilled to small models rather than piggybacking on the host backbone.

Real-world applications:

• Document and form processing pipelines, where EpiCon improves ParseBench and MP-DocVQA-style tasks by storing which document regions and layout cues matter.
• Chart, infographic, and diagram reasoning, where the memory's pointers to image regions are directly usable as evidence for the next answer.
• Visual-to-code generation (UI/screenshot-to-code), where accumulated guidance about recurring failure modes can be shared across teams working on different codebases.
• Fleet-wide service assistants, where a shared bank means a new deployment with a different model or orchestration stack can start from experience it never generated.

Industry relevance. The speed numbers matter for deployment: the 2B memory modules cut memory-operation time by 67–74% relative to backbone-sized memory models, which reduces the cost of running memory alongside expensive solvers. Because host parameters are never updated, adoption does not require retraining or fine-tuning a proprietary model — an attractive property for teams that cannot modify their base model.

Future Directions

• Reduce task-dependent negative transfer. Cross-harness evolution helped eight or nine of eleven benchmarks, but specific benchmarks regressed (for example Codex's ParseBench dropping from 63.4 to 58.9 after another harness evolved the bank). Methods for detecting or filtering lessons that hurt a particular receiving system are an open problem.
• Generalize beyond the three training domains. The trained memory models transfer to general visual-language reasoning (WorldBench gains of +3.2 to +7.8 points) but results on ReasonMap and BabyVision are inconsistent, suggesting coverage limits that further supervision could address.
• Verify tree operations more strongly. The paper states tree-operation demonstrations pass structural checks but are not individually verified through downstream MAS replay, unlike the controller updates. Extending replay verification to organization and consolidation decisions is a natural next step.
• Scale and stress-test the bank. All reported bank construction uses sets of 1,000 questions, and evaluation freezes the bank. How the bank behaves at much larger scale — and whether consolidation rules keep improving retrieval as they accumulate — is not reported.

Target Audience

Researchers and engineers working on LLM/multimodal multi-agent systems, agent memory, retrieval-augmented agents, and self-improving or self-evolving agents. It is also relevant to practitioners who want to add cross-model experience sharing to an existing agent stack without retraining a base model, and to those interested in distilling memory-management policies into small, fast, separately trained models.

Authors’ abstract

Agents can learn from past executions, but enabling different agents to reuse and build on one another's experience remains challenging. We introduce EpiCon, a shared multimodal memory framework for agent collective learning without updating host model parameters. EpiCon links question-level memory evolution to a persistent experience bank through two independently trained 2B models: a memory controller and a tree self-organizer. The controller jointly refines textual guidance and visual evidence across attempts and selectively includes visual memory. The self-organizer consolidates lessons hierarchically and retrieves experience and rules for new problems. We evaluate EpiCon on eleven benchmarks spanning four multimodal task domains, using two harnesses and multiple backbones. A frozen bank improves other systems even with a single solving attempt. A second harness raises the original system's macro-average score by 2.6 points across eleven benchmarks. Across four host configurations, EpiCon improves macro-average scores by 1.7 to 4.9 points over No Memory and reduces memory-operation time by 67\% to 74\% relative to backbone-sized memory models.