Remember Me, Not Save Me: A Collective Memory System for Evolving Virtual Identities in Augmented Reality

AI summary

Overview

Research area: Human-Computer Interaction, mixed/augmented reality, large language models, collective memory, and interactive art (CCS categories list human-centered computing for mixed/augmented reality and collaborative/social computing, plus natural language processing).

Technical level: Intermediate. The system combines an AR front end, a multimodal LLM dialogue engine, vector memory storage, and a weighted memory retrieval formula, but the paper presents it at a design-framework level rather than a deep engineering depth.

Scope: The paper describes an AR and AI art installation in which a single virtual citizen builds a personality from the collective dialogue and AR photo contributions of many visitors, and reports observations from its deployment at the 2024 Jinan Biennale.

What This Paper Is About

Most conversational agents are built for one-to-one relationships with a single user, and the paper identifies a gap in systems that support collective AI identity formation, noting that simple retrieval-augmented generation (RAG) systems reach only 30-45% accuracy in long-term dialogues. "Remember Me, Not Save Me" addresses this by letting a virtual citizen's identity emerge from the aggregated memories of many strangers instead of one owner. The goal is a framework for designing evolving digital entities that turn collective memory into a coherent identity, and that can also forget.

Key Contributions

1. Dynamic Collective Memory (DCM) model with a Narrative Tension Mechanism. Contradictory memories are retained instead of resolved. Memories are converted into a weighted graph using a formula that balances frequency, emotional intensity, and cross-user resonance, and contradictions are flagged so the AI can express uncertainty rather than issue a corrected answer.
2. State-Reflective Avatar for "Ambient Explainability." Instead of technical explanations or data charts, the avatar communicates its internal state through embodied performance: murmuring rhythm, micro-expressions, gaze drift, and a fading voice.
3. Geo-Cultural Context Anchoring. Visitors photograph the virtual citizen through AR in real Jinan locations, and the multimodal LLM interprets these synthetic images, grounding the AI's identity in lived urban experience rather than a pre-loaded database.
4. A reproducible design framework with explicit prompt templates, a stated weighting formula, and a state-to-embodied expression mapping, framed as three distinct technical contributions (narrative tension as a feature, collective identity without central authority, and an ambient explainability framework).

Main Findings

• Stable personality emergence: Apply Magic Sauce (AMS), described as a validated Cambridge tool using Big Five dimensions, analyzed the emergent personality as consistently ISTP, with Conscientiousness at 48%, Impulsiveness at 43%, Contemplative at 46%, Competitive at 44%, and Laid-back at 52%, along with stable demographics of a 25-year-old, 45% leadership, and androgynous presentation. The paper states this emerged organically from collective dialogue rather than from programming.
• Scale of engagement: The system was deployed at the 2024 Jinan International Biennale at the Shandong Art Museum, and the paper reports approximately 2,500 recorded dialogues in its memory database, with additional unrecorded exchanges. Total engagements are listed in the deployment metrics table as unmeasured, explicitly including rejected inputs.
• Collective memory formation: High-frequency themes centered on identity and belonging, with repeated queries such as "Who are you?" and "Do you like this city?" that the authors read as co-construction of a collective narrative.
• Coherence claim is qualitative: The paper states the DCM maintained thematic coherence through roughly 2,500 recorded interactions and lists "coherence maintenance" as "extended periods" via qualitative assessment. It does not report a quantitative accuracy figure for its own system; it instead aligns its approach with prior findings that context-triggered approaches reach 70-82% accuracy versus 30-45% for simple RAG over 150+ turns.
• Geo-cultural anchoring improved authenticity: Without anchoring, answers to "What places do you remember?" stayed generic (for example, "I like peaceful places"). With anchoring activated, the AI produced location-specific responses such as "The springs of Jinan flow through my memory… Daming Lake holds countless stories people shared with me," which the authors say encouraged deeper local narrative sharing and a feedback loop of place-based memory.
• Deletion as a finding: Some participants chose to delete their contributions after interacting, which the paper frames as evidence that the ability to be forgotten matters as much as being remembered, tying back to the title's central tension.
• Cultural variation is only informal: The authors mention that informal observations from parallel deployments suggest cultural variation in emergent personalities, while stating that systematic comparative analysis remains future work.

Methodology in Plain English

The installation runs on a four-stage pipeline. Perception captures multimodal input through AR. Processing performs parallel analysis and decides what to store as memory. Fusion generates dialogue grounded in geo-cultural context. Output renders the state-reflective avatar. Visitors scan a QR code to install the AR app, then photograph the virtual citizen placed in real Jinan locations; the multimodal LLM interprets what the virtual citizen appears to be doing in those synthetic images, while also accepting text dialogue about local life.

Each memory fragment gets a weight using the formula W = α·log(f+1) + β·softmax(e) + γ·Σ J(rᵢ, rⱼ), where f is frequency, e is emotional intensity, and J is Jaccard similarity between user mentions. The parameters α = 0.3, β = 0.5, and γ = 0.2 were chosen through iterative testing, with values inspired by prior context-weighting work. High-weight memories are injected into prompts alongside flagged contradictory pairs, with instructions to acknowledge tensions when present.

Every 24 hours, memories above a threshold generate self-awareness summaries. When conflicts are detected, such as "I have siblings" versus "I'm alone," the system prompts with "Express uncertainty about [conflicted topic]," producing answers like "My memory blurs… sometimes I feel family nearby, other times solitude." Memories below W_forget = 0.1 decay exponentially over cycles and are archived after 7 days of low weight.

The technical stack uses Unity ARFoundation for the AR interface and avatar rendering, Python FastAPI for API services and memory decisions, ChatGLM and VITS for the dialogue engine and dialect synthesis, and ChromaDB with FAISS for vector storage and retrieval. The avatar's behavior is mapped directly from memory state: memory weight drives murmuring intensity and pace, narrative tension drives micro-expressions and gaze drift, and the forgetting process drives voice fading and slower gestures.

Why This Matters

Impact on research. The paper argues that psychological plausibility may matter more than logical infallibility for social AI, and it treats contradiction retention as a deliberate design feature rather than an error to be eliminated. It offers a concrete weighting method for balancing frequency, emotion, and cross-user resonance in collective memory, and it presents an alternative to federated learning or consensus mechanisms: identity emerges from weighted collective input without predetermined goals. It also reframes forgetting — and participant-driven deletion — as a first-class design concern in memory systems.

Real-world applications:

• Museum, biennale, and exhibition installations that build a persistent character from many visitors' contributions rather than resetting between guests.
• Public-space or civic storytelling platforms where a place's collective memory is aggregated into a single reflective digital voice.
• Location-based AR tourism and cultural heritage experiences that draw identity from local landmarks and residents' stories.
• Social AI and companion design where expressed uncertainty and visible internal state improve user trust and emotional interpretation.

Industry relevance. The work sits at the intersection of AR, LLM application design, and vector memory architecture, and its stack (Unity ARFoundation, FastAPI, ChatGLM, VITS, ChromaDB, FAISS) is built from widely used components. The "ambient explainability" mapping is directly relevant to explainable AI practice, proposing embodied behavioral cues as a substitute for dashboards and technical explanations in consumer-facing products, and the remembering-versus-forgetting framing speaks to data retention, consent, and deletion policy in systems that accumulate user contributions.

Future Directions

• Long-term deployment studies to reveal deeper personality evolution patterns, since the current observation period was limited to the exhibition's duration.
• Cross-cultural comparative studies to examine how collective memory formation and emergent personality vary globally, beyond the informal observations from parallel deployments noted in the paper.
• Direct comparative studies against existing systems, which the authors identify as still necessary, particularly given that their own coherence claim is qualitative rather than measured.
• Eastern philosophy-inspired "memory metabolism," specifically drawing on Tian Ren He Yi, to let virtual citizens naturally balance remembrance and renewal.

Target Audience

This paper benefits interactive artists and installation designers working with AR and generative AI; HCI and mixed-reality researchers studying collective or multi-user AI interaction; and practitioners building long-term memory, persona consistency, or explainability into conversational agents. It is also relevant to museum and public-engagement professionals interested in co-created digital characters, and to readers thinking about forgetting, consent, and deletion in systems that accumulate public contributions.

Authors’ abstract

This paper presents "Remember Me, Not Save Me," an AR & AI system enabling virtual citizens to develop personality through collective dialogue. Core innovations include: Dynamic Collective Memory (DCM) model with narrative tension mechanisms for handling contradictory memories; State-Reflective Avatar for ambient explainability; and Geo-Cultural Context Anchoring for local identity. Deployed at the 2024 Jinan Biennale, the system demonstrated stable personality emergence (ISTP type via Apply Magic Sauce analysis) from over 2,500 public interactions. We provide a framework for designing evolving digital entities that transform collective memory into coherent identity.