Plural Voices, Single Agent: Towards Inclusive AI in Multi-User Domestic Spaces

AI summary

Overview

• Research area: Human-Computer Interaction (HCI), specifically agentic AI for domestic/smart-home environments, with a focus on inclusive and accessible design for children, older adults, and Neurodivergent users.
• Technical level: Intermediate. The paper mixes qualitative co-design methods with LLM fine-tuning and architecture comparison, so readers benefit from familiarity with agentic AI, LLMs, and HCI evaluation concepts (SUS, disparate impact ratio, participatory design).
• Scope in one sentence: The paper proposes the Plural Voices Model (PVM), a single-agent framework for resolving conflicting requests from multiple household members, implemented as the AgoraNest app powered by a fine-tuned Agora-4B model, and compares it against multi-agent baselines.

What This Paper Is About

Domestic AI systems such as voice assistants struggle when several household members with very different needs issue conflicting requests at the same time — for example, a child asking for bright lights and loud music while a Neurodivergent user needs dim, quiet conditions. The authors argue that multi-agent architectures (where each user type is handled by a separate specialized model) suffer from coordination failures, hallucinations, latency, and emotional harm. Their goal is a single unified agent that negotiates competing needs in real time through value alignment, adaptive safety scaffolds, and tailored communication per user type.

Key Contributions

1. The Plural Voices Model (PVM) and Agora-4B: A single-agent framework for multi-user domestic settings that performs real-time value alignment, fairness mediation, and contextual memory within one architecture, released as the open-source AgoraNest app (code and model at github.com/zade90/Agora).
2. A co-design-derived design guideline set: Six guidelines — participatory co-design for inclusive systems, transparency and user control, adaptive interaction modalities, accessible and empathetic language, ethical safeguards via the IDEAS framework (Inclusion, diversity, equity, accessibility, and safety), and human-AI partnership.
3. A synthetic multi-user dataset: 100,000+ entries generated locally with Microsoft Phi3:14b via Ollama (chosen over DeepSeek-R1:14B and Qwen 3:14b), split into 30% single-user and 70% concurrent-conflict scenarios, with roughly 10,000 entries per domain category.
4. An empirical architecture comparison: A five-stage co-design and evaluation process (N=36) comparing the unified Agora-4B system against multi-agent baselines on compliance, fairness, safety, autonomy, usability, and latency.

Main Findings

• Single-agent outperforms multi-agent on compliance and fairness (abstract figures): In preliminary evaluations, PVM reached 76% compliance vs. 70% for multi-agent baselines, 90% fairness vs. 85%, a 0% safety-violation rate vs. 7%, and better latency.
• A second, differently-scaled set of results is reported in the Introduction: structural compliance of 94% vs. 88%, fairness mediation of 82% vs. 61%, and safety violation rates of 1.5% vs. 4.2%. These figures differ from those in the abstract; the paper does not reconcile the two sets.
• Conflict handling is the core architectural difference: Figure 2 illustrates multi-agent setups producing "model confusion," hallucinations, and potential emotional harm under concurrent queries, while Figure 3 shows the Agora-nest agent using a Query Analyzer, a Reasoning & Ethical Decision block, and Possible Action Memory to resolve the same concurrent queries without conflict.
• Co-design surfaced concrete user breakdowns: Table 1 documents 14 breakdown rows across four categories (accessibility and interaction clarity, autonomy and control, multi-user coordination and fairness, safety and ethical alignment), each with participant quotes — for example, a Neurodivergent participant reporting the AI "talks too fast and uses big words," and a child reporting the AI announced a private query aloud.
• Customizable safety scaffolds were the dominant request: 75% of Neurodivergent and older users asked for slower speech and visual cues to reduce cognitive overload. Children emphasized fair turn-taking algorithms; typical adults emphasized relational privacy norms in shared households.
• Concrete design artifacts followed from the findings: a granular autonomy slider (0–100% scale), a family hub, video guidance with animations, an adaptive safety dashboard, dynamic consent pop-ups, and per-user language adaptation (step-by-step instructions for Neurodivergent users, simple wording for children, polite/formal wording for older adults).
• Model comparison against larger single-agent models: Figure 5 compares Agora-4B against LIGHT-IF 8B, Qwen2.5 7B INSTRUCT, IBM GRANITE Guardian 3.2 5B, Mellum 4B Base, and Qwen 3 4B on a child query about Paris attractions, reporting that Agora-4B gave concise, accurate answers with safety notes while some larger models produced verbose or hallucinated content.
• Latency trade-off is expected but bounded: Hypothesis H4 predicts Agora-4B incurs slightly higher latency during video-based guidance but remains under an 800 ms median threshold. The paper does not report the measured latency values in the available content.

Methodology in Plain English

The team used a five-stage process: recruit participants and obtain consent; interview users to learn individual needs; train the model on open-source datasets and build an initial prototype; run co-design workshops with conflict scenarios to refine the prototype; and validate with both human and LLM-based evaluation.

The Agora-4B model was fine-tuned from Qwen-3 4B (chosen for its small size, fast mobile response, multilingual support, and token capacity) on a curated mix of conversation datasets: MentalChat16K, SeniorTalk, autism support forums, StudyChat, EdNet, MedDialog, MoralChoice, and EmpatheticDialogues — spanning mental health counseling, caregiver-senior dialogue, pediatric education, medical Q&A, and moral dilemmas. A 2-hour virtual participatory workshop was held in February 2025 with 12 participants (3 children aged 8–12 with parental consent, 3 older adults 65+, 3 self-identified ADHD Neurodivergent individuals, and 3 typical adults), using arts-based activities for children, role-playing with voice interfaces for older adults, and adaptive prompt testing for Neurodivergent participants. Separately, 100,000+ synthetic scenarios were generated locally with Microsoft Phi3:14b through Ollama to preserve privacy around sensitive cases.

The AgoraNest app itself was built with Flask and Socket.IO for backend real-time communication, and HTML5/JavaScript plus React Native for cross-platform clients, with SpeechRecognition, pyttsx3, and the Web Audio API handling speech pipelines, JSON/CSV-based episodic memory, and GDPR-compliant data handling.

Evaluation used N=36 participants drawn from 3 families: 8 children (10–15), 6 older adults (65+), 4 Neurodivergent adults (ADHD/autism), and 18 Neurotypical users. Sessions ran 45–60 minutes and were counterbalanced between Agora-4B and multi-agent systems. Metrics included compliance, fairness via disparate impact ratio, safety, autonomy on Likert scales, usability via SUS, and latency, analyzed with paired t-tests/Wilcoxon tests and reflexive thematic analysis at a Cohen's κ ≥ 0.7 threshold. Four hypotheses (H1–H4) addressed autonomy/usability, compliance, fairness disparities, and latency respectively.

Why This Matters

Impact on research. The paper pushes back on the assumption that more specialized agents produce better multi-user experiences, arguing instead that a single agent with unified memory and ethical reasoning resolves coordination problems more reliably. It also contributes a design vocabulary — autonomy sliders, family hubs, adaptive safety dashboards, IDEAS safeguards — for inclusive domestic AI, and supplies a synthetic dataset tailored to underrepresented user archetypes.

Real-world applications:

• Smart-home assistants that must arbitrate simultaneous, conflicting requests from family members with different sensory and accessibility needs.
• Eldercare and assisted-living systems that combine health monitoring, medication reminders, and fall detection with step-by-step confirmation flows and caregiver alerts.
• Assistive technology for Neurodivergent users, including task decomposition, distraction reduction, and sensory-adjusted environments for ADHD and autism spectrum conditions.
• Child-facing educational and companion agents that enforce parental oversight, fair turn-taking, and age-appropriate language without encouraging over-dependence.

Industry relevance. The comparison against Mellum 4B, Qwen2.5 7B, LIGHT-IF 8B, and IBM Granite Guardian 3.2 5B positions the work against production-grade small models, and the claim that Agora-4B matches or beats models more than twice its size speaks directly to on-device deployment constraints where latency, memory, and privacy matter. The emphasis on local processing and GDPR-compliant handling also maps onto regulatory pressure on consumer AI.

Future Directions

• Reconcile and extend the evaluation numbers. The abstract and the introduction report different compliance, fairness, and safety figures (76%/70%, 90%/85%, 0%/7% versus 94%/88%, 82%/61%, 1.5%/4.2%); the paper does not specify whether these are different metrics, phases, or samples, and clarifying this is a clear next step.
• Report full quantitative outcomes. Latency measurements, SUS scores, autonomy Likert results, disparate impact ratios, and the actual Cohen's κ value are not reported in the available content, so completing that reporting is needed to substantiate H1–H4.
• Run longitudinal and larger-scale studies. The paper itself notes that long-term studies and human-in-the-loop designs are essential for aligning ethical principles with practical deployment; the current evaluation covers 36 participants from 3 families in 45–60 minute sessions.
• Address over-reliance and cognitive offloading. Cited concerns about reduced critical thinking in children and dependency risks for Neurodivergent and older users point toward designs that gradually reduce support, as the paper's strength-focused co-design discussion suggests.
• Harden against security and privacy threats. The background raises "memory poisoning," digital-twin control loss, and data-sharing risks in multi-agent homes, all of which remain open problems for a persistent-memory single agent.

Target Audience

• HCI and accessibility researchers studying inclusive design, participatory co-design, and multi-user interaction.
• AI/ML engineers building on-device or privacy-preserving LLM agents for consumer hardware, especially those weighing single-agent versus multi-agent architectures.
• Product designers and builders of smart-home, eldercare, and child-facing assistants looking for concrete design patterns like autonomy sliders, family hubs, and adaptive safety dashboards.
• Policy and ethics reviewers interested in how IDEAS-style safeguards, GDPR compliance, and consent mechanisms are operationalized in domestic AI.

Authors’ abstract

Domestic AI agents faces ethical, autonomy, and inclusion challenges, particularly for overlooked groups like children, elderly, and Neurodivergent users. We present the Plural Voices Model (PVM), a novel single-agent framework that dynamically negotiates multi-user needs through real-time value alignment, leveraging diverse public datasets on mental health, eldercare, education, and moral reasoning. Using human+synthetic curriculum design with fairness-aware scenarios and ethical enhancements, PVM identifies core values, conflicts, and accessibility requirements to inform inclusive principles. Our privacy-focused prototype features adaptive safety scaffolds, tailored interactions (e.g., step-by-step guidance for Neurodivergent users, simple wording for children), and equitable conflict resolution. In preliminary evaluations, PVM outperforms multi-agent baselines in compliance (76% vs. 70%), fairness (90% vs. 85%), safety-violation rate (0% vs. 7%), and latency. Design innovations, including video guidance, autonomy sliders, family hubs, and adaptive safety dashboards, demonstrate new directions for ethical and inclusive domestic AI, for building user-centered agentic systems in plural domestic contexts. Our Codes and Model are been open sourced, available for reproduction: https://github.com/zade90/Agora