The Interaction Tax: When Communication Erases Diversity in Multi-Agent Teams

AI summary

Overview

Research area: Multi-agent large language model (LLM) systems for scientific optimization, with connections to ensemble diversity theory and AI-assisted research workflows.

Technical level: Intermediate. The paper assumes familiarity with LLM prompting workflows (debate, critique loops, sampling) and basic benchmarking concepts such as confidence intervals and bootstrap tests, but its core argument is conceptual rather than mathematically heavy.

Scope in one sentence: The paper isolates model diversity as a controlled variable across 11 verifier-scored optimization tasks to test when multi-agent interaction preserves versus destroys the diversity that makes multi-model teams useful.

What This Paper Is About

Published work disagrees about whether multi-agent LLM interaction helps: debate, critique loops, and mixture-of-agents synthesis report gains, while other studies find interaction adds cost without quality improvements under equal budgets. The authors argue this contradiction comes from conflating different kinds of communication. Their specific claim is that when agents read each other's complete candidate solutions, proposals converge within a single round, erasing the very diversity that motivated using multiple model families in the first place — a cost they name the interaction tax.

Key Contributions

1. A named failure mode: The paper identifies and characterizes the "interaction tax," the loss of useful diversity when agents exchange full candidate solutions, showing that this collapse occurs within one round of full-output exchange rather than gradually.
2. A controlled diversity experiment: A 2×2 factorial crossing model diversity (same vs. diverse proposers) with synthesis (MoA synthesis vs. best-score selection) that isolates diversity as the only factor whose confidence interval excludes zero (diversity coefficient +0.188, CI [+0.073, +0.299], p<0.001; synthesis coefficient −0.010, CI [−0.111, +0.094]).
3. A boundary condition for critique: Evidence that critique reverses the tax only when the violated rule is easy for a model to locate and repair — Knapsack-50 (diverse Debate 10/10 feasibility vs. 2/10 same-model) versus 3AP-Free-100 (diverse Debate 0/10 vs. 6/10 same-model).
4. A reproducible artifact and workflow: A release containing 1,556 saved JSON run traces, verbatim prompts for all 10 configurations, both visible and hidden evaluators, and analysis scripts, organized into a five-step practical workflow for multi-agent optimization.

Main Findings

• Same-model teams have coverage gaps; diverse teams do not. Every same-model team scores Q = 0 on at least one task, while a diverse team (Claude + GPT-4o + Gemini) never scores zero on any task, even though its aggregate Q is comparable to the best same-model team.
• Different model families win on different tasks. In the six tasks where scores diverge, Claude wins Circle Packing (Q = 1.000) and TSP-50 (Q = 0.141), GPT-4o wins Erdős (Q = 0.710) and TSP-100 (Q = 0.021), and Gemini wins DiffBases (Q = 0.610) and Flat Poly (Q = 0.143).
• Full-solution interaction reverses sign when models are diverse. Chain goes from +0.051 same-model MIG to −0.024 diverse MIG (P(same > div) = 97%), MAgICoRe from +0.044 to −0.035 (99%), and Debate from +0.012 to −0.078 (88%). HPE is negative in both conditions (−0.163 same, −0.145 diverse; P = 38%).
• MoA escapes the tax. MoA, the only configuration where proposers never read each other's outputs, keeps positive MIG across both conditions (+0.012 same-model, +0.016 diverse; P = 42%), and it is the only configuration whose confidence interval for aggregate MEG includes zero.
• No configuration achieves positive aggregate MEG over every single-agent baseline across the ten configurations and nine tasks plotted.
• Diversity collapses immediately, not gradually. Mean pairwise distance between agent solutions falls from 0.315 before interaction to 0.229 after, and on Erdős, diverse Debate regresses at round 3 after scoring strongly at round 2 when only critiques were exchanged.
• Synthesis rarely recombines. On five of seven tasks, synthesis copies the best proposer's output at least 80% of the time; on Difference Bases it degrades the result 50% of the time.
• A degenerate constant can dominate and erase diversity. GPT-4o outputs the identical trivial constant on 100% of Erdős seeds (Q = 0.710), so any interaction or synthesis step can converge to that point regardless of what other backbones proposed.
• Critique helps only for locally checkable faults. On graded-feedback tasks, the first critique round degrades solutions 57% of the time (17/30 runs). On verifiable constraint tasks, MAgICoRe raises feasibility from 0% for Best-of-N to 47% on Knapsack-50 (p = 0.003) and 73% on 3AP-Free-100 (p<0.001).
• Visible and hidden evaluators diverge. Visible and hidden rankings disagree on three of the nine optimization tasks analyzed with hidden scores (MaxCut ρ = 0.891 with top-1 inverted, DiffBases ρ = 0.806, MolQED ρ = 0.952).
• Diversity only helps where models differ more from each other than from themselves. The between/within-model variance ratio is 15.2 for Circle Packing and 2.0 for MolQED ("yes"), 1.3 for Flat Poly and 1.2 for Erdős ("marginal"), and below 1 for TSP-100 (0.5), MaxCut (0.3), and DiffBases (0.2). The reported diversity effect is also task-dependent: removing Erdős drops the coefficient to +0.014 with a CI crossing zero.

Methodology in Plain English

The researchers assembled 11 optimization tasks — including Circle Packing, Difference Bases, Erdős Overlap, Flat Polynomials, TSP-50, TSP-100, MaxCut, LJ-n=41, Molecule QED, Knapsack-50, and 3AP-Free-100 — four of which are adapted from the AlphaEvolve suite with smaller parameterizations. Every task exposes the same interface: an agent returns a JSON candidate solution and a deterministic verifier returns a scalar score. Agents can see a development ("visible") evaluator while they search, and the final chosen artifact is scored once by a stricter hidden evaluator. Raw scores are normalized to Q in [0, 1].

They implemented ten configurations from a larger protocol pool, all built on Claude Sonnet 4, GPT-4o, and Gemini 2.5 Flash, all running under the same budget vector (T=200K, W=600s, C=30s, K=25). Four are single-agent baselines (Single-Shot, Best-of-N with N = 8, Self-Refine, Verifier-Guided Search); the rest are multi-agent workflows covering sequential revision (Homo-Chain, Cross-Chain), solver-reviewer-refiner critique (MAgICoRe), critique exchange (Debate), plan-and-execute (HPE), and independent-proposer synthesis (MoA). Proposer calls use temperature 0.7; critique and synthesis calls use 0.3–0.5.

The key measurement is MIG (does interaction beat independent generation?) and MEG (does a configuration beat every single-agent baseline?). The main benchmark uses five seeds per cell, and a separate 2×2 factorial over three tasks with sufficient score variance uses ten seeds per cell (N = 120) with a 10,000-round task-stratified bootstrap. Two tasks, MaxCut and LJ-n=41, are dropped from most analyses because every configuration scores Q = 0 there. The 2×2 design deliberately crosses whether proposers are same-model or diverse against whether their outputs are synthesized or simply selected by best score.

Why This Matters

Impact on research. The paper reframes the multi-agent debate in the literature as an information-exposure question rather than an agent-count question. Previous negative results for multi-agent systems used same-model teams or did not control for model diversity; by isolating diversity as a variable, this work supplies a mechanism — convergence at the moment of full-solution exchange — that can explain why debate, critique, and synthesis results have been inconsistent. It also connects LLM multi-agent behavior to the classical ensemble diversity decomposition, where ensemble error equals mean member error minus a diversity term that shrinks toward zero as members converge.

Real-world applications.

• AI-assisted mathematical and algorithmic discovery, where the recommended workflow is to pick 2–3 diverse model families, generate candidates independently, rank them with a deterministic checker, and only add critique where violations are locally checkable.
• Drug-like molecule design, illustrated by a Molecule QED run where three independent proposers produced scores of 0.828, 0.685, and 0.547 and the synthesizer selected the top molecule (dev score 0.828, hidden score 0.738).
• Combinatorial logistics and routing, such as TSP-50/TSP-100 tour construction, where different backbones produce different quality tours.
• Constraint-satisfaction pipelines like Knapsack-50, where feasibility verification is exact and critique passes can repair capacity violations.

Industry relevance. Any team running several LLMs in a pipeline pays per call, and the paper's central practical claim is that spending budget on full-solution exchange is a weak default. The recommended pattern — parallel diverse generation plus selection, with critique reserved for checkable failures — is directly actionable, cheap to implement, and the artifact reproduces its tables from cached results in about 3 minutes without an API key.

Future Directions

1. Find the best lower-bandwidth channel. The authors explicitly note they identify full-solution interaction as a weak default but do not directly test which alternative works best. They propose a sharing ablation over scores, method descriptions, failure causes, verified summaries, and late crossover.
2. Test whether the tax transfers to open-ended tasks. All tasks here are verifier-scored optimization problems, so the authors state the interaction tax may not carry over unchanged to writing, planning, or dialogue.
3. Broaden the statistical base. The main benchmark uses only five seeds per condition, and the diversity factorial uses ten seeds on only three tasks; only two hard-validity tasks (Knapsack and 3AP-Free) support the feasibility analysis, even though TSP also has hard structural constraints.
4. Understand and mitigate degenerate backbones. On Erdős, GPT-4o and Gemini each produced a single stereotyped output across all seeds, so a nominal three-backbone team had effectively one contributor. Determining when a backbone contributes genuine strategy variance — rather than just a different label — is an open problem the paper frames but does not solve.

Target Audience

Researchers and practitioners working on multi-agent LLM systems, LLM ensembles, and AI-assisted scientific optimization will get the most from this paper, especially those deciding how much inter-agent communication to allow in a pipeline. It is also relevant to ensemble-learning researchers interested in whether classical diversity decomposition carries over to language model teams, and to engineers who need a concrete, budget-matched workflow recipe rather than a headline claim about adding more agents.

Authors’ abstract

Does multi-agent LLM interaction help or hurt? Some work reports gains from debate (Du et al., 2024), critique loops (Chen et al., 2025), and mixture-of-agents synthesis (Wang et al., 2025), while other work finds that interaction adds cost without improving quality under equal budgets (Tran &amp; Kiela, 2026; Xu et al., 2026; Jarrett et al., 2025), or that independent sampling already captures multi-agent gains (Li et al., 2024). We argue this contradiction partly reflects a missing distinction, because not all multi-agent communication is equal. Different model families find structurally different solutions, but when agents read each other's complete outputs, their proposals converge within one round, erasing the diversity that motivates using multiple models. We call this the interaction tax. We test 11 verifier-scored optimization tasks under matched budgets and find that full-solution interaction is a weak default. Independent proposal generation avoids this collapse. Full-solution interaction mainly makes agents stay close to the first solution they see instead of trying different approaches, and critique helps only if the violated rule is easy for the LLM to find and fix. These results suggest that multi-agent performance depends less on the number of agents than on the information they exchange, and interaction helps only when agents share the right information at the right time.