Adaptive Multi-Agent Response Refinement in Conversational Systems

AI summary

Overview

• Research area: Natural Language Processing, specifically multi-agent large language model (LLM) systems for response refinement in multi-turn conversational systems.
• Technical level: Intermediate. The paper assumes familiarity with LLM prompting, conversational datasets, and automatic evaluation with G-Eval, but the method itself is presented conceptually with formal notation kept simple.
• Scope: The paper introduces MARA (Multi-Agent Refinement with Adaptive agent selection), a framework that refines an LLM's conversational responses using three specialized refining agents and a planner agent that dynamically decides which agents to use, and in what order, for each query.

What This Paper Is About

LLMs in conversational systems sometimes produce responses that misalign with a user's persona, miss required factual knowledge, or lose coherence with earlier turns, and asking users to detect these errors and request corrections interrupts the conversation. The paper's goal is to refine the response automatically before it reaches the user. Instead of having one LLM critique and rewrite its own output, the authors assign separate agents to factuality, personalization, and coherence, and add a planner agent that adaptively selects and sequences which refining agents a given query needs.

Key Contributions

1. A multi-agent refinement framework with three specialized refining agents. MARA defines a fact-refining agent (ensuring factual accuracy), a persona-refining agent (aligning responses with the user's profile), and a coherence-refining agent (maintaining coherence across conversational turns), each instantiated with its own role-specific prompt template.
2. A dynamic communication strategy driven by a planner agent. Rather than a fixed sequence or a simultaneous-then-finalize pipeline, a planner agent outputs the sequence of refining agents required for each query together with justifications, and each refining agent receives those justifications along with the response refined by the preceding agent.
3. Validation across persona-focused, knowledge-focused, and combined conversational settings. The framework is evaluated on PersonaChat, INSCIT, and FoCus, and additionally on the role-playing dataset PRODIGy and the domain-specific Ubuntu Dialogue Corpus.
4. Analysis of why role specialization and adaptive sequencing matter. Ablations over individual refining agents, a single-agent variant that merges all three perspectives, an iterative single-agent variant, random versus ideal planner assignments, and comparison of simultaneous versus fixed sequential communication.

Main Findings

• MARA outperforms all baselines on the three main datasets using Claude. In Table 1, MARA reaches an Overall score of 62.00 on PersonaChat, 72.61 on INSCIT, and 74.51 on FoCus, compared with No Refine at 34.98, 65.07, and 56.71 respectively. Among baselines, Self-Refine scores 58.41 / 57.83 / 47.07, SPP 45.55 / 65.74 / 60.47, LLMvLLM 12.57 / 51.59 / 30.92, MADR 23.21 / 52.89 / 34.71, and MultiDebate 34.70 / 61.70 / 54.81.
• Single-agent refinement fails on knowledge-heavy tasks. On PersonaChat, Self-Refine and SPP beat the no-refinement baseline, but on INSCIT and FoCus single-agent refinement degrades performance, particularly in groundedness.
• Per-metric results for MARA (Claude). PersonaChat: Coherence 2.32, Groundedness 0.56, Naturalness 1.98, Engagingness 2.54. INSCIT: 2.62, 0.59, 2.26, 2.74. FoCus: 2.67, 0.65, 2.15, 2.83.
• Using a stronger model only for the fact-refining agent helps. MARA, in which the fact-refining agent uses Claude Sonnet 3.5 while the responding agent uses Claude Sonnet 3, scores higher than MARA* (fact-refining agent also Sonnet 3) in every dataset: PersonaChat 62.00 vs 60.24, INSCIT 72.61 vs 67.79, FoCus 74.51 vs 68.77.
• The planner's adaptive selection beats random assignment and approaches an ideal planner. On 20 conversations from FoCus (Table 2), No Refine scores Coherence 2.21, Groundedness 0.40, Naturalness 1.88, Engagingness 2.05 with 1.0 agent; MARA with a random assignment scores 2.42, 0.47, 1.95, 2.64 with 2.0 agents; MARA scores 2.54, 0.58, 2.07, 2.76 with 4.4 agents; and an ideal planner that brute-forces optimal sequences scores 2.88, 0.80, 2.58, 2.93 with 3.0 agents.
• Agent distribution varies by dataset. The persona-refining agent dominates where user preferences matter, the fact-refining agent dominates in knowledge-focused datasets, and both are allocated in similar proportions for the dataset requiring both.
• MARA generalizes to role-play and domain-specific dialogue. On PRODIGy with GPT as the base model, MARA's Overall score is 63.00 versus No Refine at 52.50, Self-Refine 52.13, SPP 45.63, LLMvLLM 21.28, MADR 40.75, and MultiDebate 44.00. On the Ubuntu Dialogue Corpus with Claude, MARA scores 68.88 versus No Refine 57.57, Self-Refine 46.38, SPP 51.50, LLMvLLM 20.33, MADR 44.78, and MultiDebate 41.38.
• Human evaluation agrees with G-Eval. Eight English-fluent participants evaluated 288 conversational queries from FoCus. MARA scored an Overall of 82.88, against No Refine 65.50, SPP 64.63, Self-Refine 44.38, and MADR 43.50. Spearman correlations between G-Eval and human judgments were 0.51 for coherence, 0.48 for groundedness, 0.35 for naturalness, and 0.58 for engagingness.
• Every refining agent contributes. FoCus ablations (Table 6): No Refine 56.75; with fact 68.75; with persona 68.38; with coherence 62.25; a single agent integrating all three aspects 67.50; a single agent with iteration 61.20; full MARA 74.38. The iterative single-agent variant performing worse than the one-shot single agent suggests initial errors are amplified through repeated iterations.
• Sequential order matters, which motivates dynamic selection. On FoCus (Table 7), simultaneous communication with a finalizer scores 2.57 / 0.59 / 1.93 / 2.75 using 5.0 agents; static sequential orders yield 2.54 / 0.56 / 1.99 / 2.85, 2.65 / 0.60 / 2.15 / 2.66, and 2.56 / 0.59 / 1.96 / 2.68, each using 4.0 agents; MARA reaches 2.67 / 0.65 / 2.15 / 2.83 with 4.4 agents.
• Refinement helps even when the responding model is already strong. In Table 8, No Refine with Sonnet 3.5 scores 2.50 / 0.58 / 2.10 / 2.32, while MARA with Sonnet 3.5 for both scores 2.87 / 0.78 / 2.55 / 2.91.
• MARA works across model families. On FoCus (Table 9), GPT-4o-mini No Refine scores 2.45 / 0.47 / 2.27 / 1.98 versus MARA 2.61 / 0.57 / 2.32 / 2.70; LLaMA 3.1 8B 2.30 / 0.39 / 2.09 / 1.88 versus 2.46 / 0.48 / 2.17 / 2.24; LLaMA 3.1 70B 2.41 / 0.44 / 2.25 / 1.88 versus 2.60 / 0.55 / 2.31 / 2.67.

Methodology in Plain English

A responding LLM first produces an initial reply to the user's query given the conversational history. MARA then attempts to improve that reply before it is shown.

Three refining agents exist, each a prompted LLM with a single job: check facts and correct hallucinations, align the reply with what is known about the user, or fix coherence with earlier turns. A planner agent looks at the query and the initial response and decides which of these refining agents are needed and in which order, and it also states reasons for its choices. The chosen agents then run one after another, each taking the previous agent's refined response plus the planner's justifications as input.

The authors compare three communication strategies. In simultaneous communication all three refining agents work independently on the initial response and a separate finalizer agent merges their outputs. In sequential communication the agents run in a fixed order, and the paper tests three such orders. In the dynamic strategy, the planner chooses the order per query.

Refined responses are scored with G-Eval using four metrics: Coherence (1–3), Groundedness (0–1), Naturalness (1–3), and Engagingness (1–3), plus an Overall score that is a scale-normalized average of the four. The G-Eval judge is GPT-4o mini. The main experiments use Claude Sonnet 3 as the base model, with Claude Sonnet 3.5 assigned to the fact-refining role; a variant called MARA* keeps Sonnet 3 for every agent. Additional runs use GPT-4o-mini, LLaMA 3.1 8B, and LLaMA 3.1 70B. For each dataset the authors sample 100 conversations, giving 673 queries for PersonaChat, 506 for INSCIT, and 563 for FoCus. FoCus averages 11.9 conversational turns, PersonaChat 14.0, and INSCIT 11.8.

Why This Matters

The work argues that response quality in conversational systems should be improved proactively, without waiting for a user to notice an error and ask for a retry. It also provides evidence that dividing refinement labor across specialized agents is more effective than asking one model to critique itself on many aspects at once, and that the right division of labor is query-dependent.

Real-world applications:

• Personalized assistants and chatbots that must keep responses consistent with a known user profile over long conversations.
• Information-seeking and knowledge-grounded dialogue systems where responses are grounded in a source such as Wikipedia and hallucination is costly.
• Role-playing and character-driven conversational agents, where PRODIGy results suggest the approach transfers even though the objective differs from user-persona alignment.
• Domain-specific technical support dialogue, as tested with the software-problem-solving Ubuntu Dialogue Corpus.

Industry relevance: the framework is unsupervised and prompt-based, so it can be layered on top of existing deployed LLMs without retraining. The paper also shows that a smaller open-source model (LLaMA 3.1 8B) benefits from the refinement pipeline, and it explicitly discusses resource efficiency as a deployment consideration, including per-query agent access counts.

Future Directions

• Improving the planner agent. The gap between the current unsupervised planner and the ideal planner in Table 2 suggests constructing a dataset with labeled ideal sets and sequences of required perspectives and fine-tuning the planner on it.
• Scalability and resource efficiency. The authors flag multi-agent overhead as a concern and note that a better planner reduces LLM accesses; exploring lightweight or more efficient agent models is raised as a next step.
• Integrating external tools and knowledge sources. The framework's flexibility in assigning different LLMs per role opens the possibility of incorporating Retrieval-Augmented Generation (RAG) systems.
• Safety and ethical safeguards. The ethics statement proposes adding an agent that monitors both user inputs and generated content for harmful or offensive material.

Target Audience

Researchers and practitioners working on conversational AI, LLM self-refinement, and multi-agent LLM orchestration. It is also relevant to engineers building production dialogue systems who need automatic post-generation quality control, and to readers interested in evaluation methodology, given the reported comparison between G-Eval and human ratings (including the notably lower Spearman correlation of 0.35 for naturalness).

Authors’ abstract

Large Language Models (LLMs) have demonstrated remarkable success in conversational systems by generating human-like responses. However, they can fall short, especially when required to account for personalization or specific knowledge. In real-life settings, it is impractical to rely on users to detect these errors and request a new response. One way to address this problem is to refine the response before returning it to the user. While existing approaches focus on refining responses within a single LLM, this method struggles to consider diverse aspects needed for effective conversations. In this work, we propose refining responses through a multi-agent framework, where each agent is assigned a specific role for each aspect. We focus on three key aspects crucial to conversational quality: factuality, personalization, and coherence. Each agent is responsible for reviewing and refining one of these aspects, and their feedback is then merged to improve the overall response. To enhance collaboration among them, we introduce a dynamic communication strategy. Instead of following a fixed sequence of agents, our approach adaptively selects and coordinates the most relevant agents based on the specific requirements of each query. We validate our framework on challenging conversational datasets, demonstrating that ours significantly outperforms relevant baselines, particularly in tasks involving knowledge or user's persona, or both.