The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMs

AI summary

Overview

• Research area: Multi-agent communication (MA-Comm) across three paradigms — multi-agent reinforcement learning (MARL), emergent language (EL), and large language model (LLM)-based multi-agent systems.
• Technical level: Intermediate. The paper is a survey rather than a new algorithm, and it assumes some familiarity with reinforcement learning, partially observable settings, and language-model agents, while grounding modern methods in speech-act theory and game-theoretic equilibrium concepts.
• Scope: A unified, systematic survey that organizes multi-agent communication research by the "Five Ws" — who communicates with whom, what is communicated, when communication occurs, why it is needed, and how it is motivated and operationalized.

What This Paper Is About

Multi-agent systems in the real world, from autonomous vehicles to collaborative AI assistants, operate under partial observability, so agents must exchange information to reduce uncertainty and coordinate. Research on how agents should communicate has split into separate communities: MARL (where messages are learned signals optimized for reward), emergent language (where structured protocols arise from interaction), and LLM-based systems (where agents communicate in natural language inherited from pretraining). The paper's goal is to give these three lines of work one shared vocabulary, showing how each paradigm emerged in response to the limitations of the previous one and how they address complementary aspects of the same problem.

Key Contributions

1. A unified survey across paradigms. The authors state this is the first survey that systematically unifies MARL-based communication, emergent language, and LLM-powered multi-agent systems, rather than treating communication as a subsection of MARL.

2. A Five Ws analytical framework. The literature is organized around who communicates with whom, what is communicated, when communication occurs, why communication is needed, and how it is motivated and operationalized — a consistent lens applied across all three paradigms.

3. Cross-paradigm bridging analysis. Dedicated bridging subsections and a comprehensive Bridge section explain how limitations in MARL communication motivated EL, and how gaps in both motivated LLM-based and hybrid LLM–MARL systems.

4. Foundational, formal, and game-theoretic grounding, plus open challenges. The survey connects modern methods to classical views of communication as action, adds concise mathematical formalizations of communication structures, links mixed-motive and competitive settings to Nash and Bayesian Nash equilibrium concepts, and identifies open problems in grounding, interpretability, generalization, efficiency, and theoretical guarantees.

Main Findings

• Communication is best viewed as action, not information transfer. Speech act theory (Austin, Searle) and the work of Grice and Wittgenstein framed utterances as goal-directed acts. The survey argues that MARL message actions preserve this insight — a message is a decision variable chosen to influence others — but replace explicit pragmatic conventions with reward-driven learning.

• An evolutionary progression, not competing alternatives. MARL communication was motivated by partial observability and coordination; EL was motivated by the opacity and environment-specific brittleness of learned MARL messages; LLM-based communication was motivated by EL's need for training from scratch and its grounding, generalization, and scalability challenges.

• A structured comparison of foundational assumptions. Table 1 contrasts classical AI communication (explicit, hand-defined, logic-based semantics; high interpretability; poor scalability), MARL communication (learned actions optimized for reward; implicit semantics; weak generalizability beyond co-trained teams; low-to-moderate interpretability), and LLM-based communication (language generation conditioned on prompts; semantics inherited from pretraining; weak grounding; high surface-level interpretability but uncertain control guarantees). Hybrid LLM–MARL systems are positioned as a middle ground balancing expressiveness, grounding, and control.

• Centralized training with decentralized execution is the recurring MARL pattern. Early methods such as CommNet and IC3Net aggregate hidden states across agents, while later methods such as TarMAC (attention-based message routing) and DICG (graph-structured message propagation) let agents dynamically choose communication partners.

• A gap in the existing survey literature. Table 2 compares 26 prior surveys across MARL, emergent language, and LLM categories on four dimensions — communication focus, LLM focus, emergent language, and systematic framework — and only the authors' own survey is marked as covering all of them.

• Discrete and constrained messages improve interpretability but create new problems. EL approaches using one-hot messages, binary signals, and other constrained formats make communication more human-readable, but reveal coordination failures such as zero-shot breakdowns where independently trained agents cannot understand each other's learned protocols.

• LLM agents shift communication from task-specific signaling to a reusable interface. Because LLMs bring linguistic structure, commonsense reasoning, and world knowledge from pretraining, agents can coordinate with zero-shot or few-shot interaction instead of learning protocols from scratch. Proposed LLM communication architectures include direct messaging, chain-of-thought interactions, hierarchical structures, and graph-based exchanges.

• No empirical benchmarks are reported. As a survey, the paper reports no experiments, datasets, or quantitative results of its own; the content provided does not report performance figures.

Methodology in Plain English

The authors conducted a systematic literature review rather than an experimental study. They searched for work published before 2025 using Google Scholar with keyword combinations such as "multi-agent communication," "MARL communication," "emergent language," "signaling games," "multi-agent LLM," and "language-based coordination." They also manually examined proceedings of major AI and machine learning venues — NeurIPS, ICML, ICLR, AAAI, and AAMAS — along with relevant workshops, and used backward and forward citation tracking to find influential papers that keyword search missed. The stopping criterion for each search term was a full results page without any new relevant papers. A small number of earlier seminal works, such as Dec-POMDPs, classical team decision theory, and communication-constrained control, were included to provide foundational context.

Papers were included if they (1) treat communication between agents as a core part of the problem or solution rather than an implementation detail, (2) involve multiple decision-making agents in cooperative, competitive, or mixed-motive settings, and (3) present a methodological contribution, empirical evaluation, or formal framework related to communication learning, representation, or usage. The fourth inclusion criterion concerns peer-reviewed publication status; the text of this criterion is cut off in the content provided. After selection, the literature was sorted into the three paradigms and, within each, categorized by the Five Ws.

Why This Matters

Impact on research. The three research communities that study agent communication have largely developed in isolation, with each new paradigm emerging in response to limitations the previous one could not solve. By supplying one shared taxonomy and explicit bridging sections, the survey gives researchers a way to compare methods across paradigms, recognize which design problems have already been solved elsewhere, and build hybrid systems that combine learning, language, and control. It also connects longstanding questions about grounding, belief alignment, and coordination under uncertainty to their classical roots in philosophy of language.

Real-world applications named in the paper:

• Robotics, including navigation and manipulation, where multiple robots must coordinate with limited local views.
• Autonomous systems, including autonomous driving, where vehicles must share information under real-time and bandwidth constraints.
• Smart grid control and other fully cooperative multi-agent infrastructure settings.
• Collaborative AI assistants and human-AI interaction, where natural-language communication with people is required.
• Strategic and adversarial environments such as StarCraft and Dota, where signaling, misinformation, and negotiation matter.

Industry relevance. The survey stresses practical constraints that practitioner-facing systems face: real communication networks are discrete and bandwidth-limited, whereas much MARL research assumes continuous real-valued messages. Interpretability, generalization beyond co-trained teams, and grounding in environment dynamics are identified as the main barriers to deployment, and the paper positions hybrid LLM–MARL designs as a practical path toward systems that are simultaneously expressive, grounded, and controllable.

Future Directions

• Grounding, interpretability, and generalization. The survey identifies these as core open problems spanning all three paradigms, including the difficulty of grounding LLM communication in environment dynamics and of generalizing learned protocols beyond the specific team and task they were trained on.

• Efficiency and theoretical guarantees. Communication overhead, bandwidth constraints, and scalability remain unresolved, and the paper argues for stronger theoretical grounding, building on classical Nash and Bayesian Nash equilibrium concepts for mixed-motive and competitive settings.

• Benchmarking. The authors call for progress on benchmarking as a research direction for the field, noting in their review of related work that prior MARL communication reviews identified a lack of standardized environments as a gap.

• Hybrid LLM–MARL systems and human-centric communication. The survey points toward frameworks that combine learned behaviors, emergent structure, and pretrained language, with attention to safe and human-aligned communication in increasingly open-ended and safety-critical settings.

Target Audience

This survey is most useful for researchers and graduate students working on multi-agent reinforcement learning, emergent communication, or LLM-based multi-agent systems who want a single entry point spanning all three areas. It also serves practitioners building multi-robot, autonomous-driving, or collaborative-assistant systems who need to understand the trade-offs among learned protocols, discrete emergent languages, and natural-language agents, and researchers in NLP or game theory looking for how their formal tools connect to modern learning-based coordination.

Authors’ abstract

Multi-agent sequential decision-making powers many real-world systems, from autonomous vehicles and robotics to collaborative AI assistants. In dynamic, partially observable environments, communication is often what reduces uncertainty and makes collaboration possible. This survey reviews multi-agent communication (MA-Comm) through the Five Ws: who communicates with whom, what is communicated, when communication occurs, and why communication is beneficial. This framing offers a clean way to connect ideas across otherwise separate research threads. We trace how communication approaches have evolved across three major paradigms. In Multi-Agent Reinforcement Learning (MARL), early methods used hand-designed or implicit protocols, followed by end-to-end learned communication optimized for reward and control. While successful, these protocols are frequently task-specific and hard to interpret, motivating work on Emergent Language (EL), where agents can develop more structured or symbolic communication through interaction. EL methods, however, still struggle with grounding, generalization, and scalability, which has fueled recent interest in large language models (LLMs) that bring natural language priors for reasoning, planning, and collaboration in more open-ended settings. Across MARL, EL, and LLM-based systems, we highlight how different choices shape communication design, where the main trade-offs lie, and what remains unsolved. We distill practical design patterns and open challenges to support future hybrid systems that combine learning, language, and control for scalable and interpretable multi-agent collaboration.