Enabling Agents to Communicate Entirely in Latent Space

AI summary

Overview

Research area: Multi-agent LLM systems and latent-space (continuous representation) communication.

Technical level: Intermediate to Advanced. The paper assumes familiarity with transformer hidden states, supervised fine-tuning, and divergence-based training objectives, though its core idea is explained conceptually.

Scope: This paper proposes Interlat (Inter-agent Latent Space Communication), a feasibility study of enabling LLM-based agents to exchange last-layer hidden states directly rather than natural-language tokens, and to compress those states through latent-space reasoning.

What This Paper Is About

LLM-based agents normally talk to each other in natural language, which forces each agent to squeeze its rich internal state into a sequence of discrete tokens, discarding alternative reasoning paths and adding redundancy. The authors ask whether agents can instead transmit their "thoughts" directly as continuous hidden states, and whether that communication can be compressed into a few latent steps without losing task-relevant information. They build and test a two-agent sender (reasoning agent) / receiver (actor agent) setup to see if this "latent communication" is feasible.

Key Contributions

1. A latent communication paradigm (Interlat). Instead of transmitting decoded tokens, the reasoning agent transmits the temporally aligned last-layer hidden states corresponding to its generated message, which are fed into the receiving agent's input embedding sequence, bracketed by <bop> and <eop> special tokens and processed by a trainable lightweight self-attention plus projection layer called a communication adapter.

2. A supervised training objective for interpreting latents. The total loss combines a task cross-entropy loss, a conditional thought separation loss (a weighted Jensen–Shannon divergence between output distributions conditioned on matched latents versus mismatched latents sampled from a different task), and a plan-aligned regulation loss (KL divergence plus a cosine term against the distribution induced by the corresponding language-space plan), stabilized by a stochastic token-to-latent mixing curriculum with replacement rate r in {0, 0.1, ..., 1.0}.

3. An information-preserving compression method. A separate reasoning model M_φ autoregressively generates compact latent messages of length K steps entirely in latent space, trained with a composite objective of task loss, an uncertainty-weighted agreement loss (weighted by entropy differences relative to a no-latent baseline), and a latent geometry alignment loss, while the actor and its adapter stay frozen.

4. Empirical validation across backbones, benchmarks, and topologies. Evaluation on Alfworld and MATH with Qwen2.5-7B-Base, Qwen2.5-0.5B-Base, and LLaMA3.1-8B-Base, including cross-family latents (Qwen2.5-7B latents training a LLaMA3.1-8B actor) and 3-agent sequential chain and parallel tree topologies.

Main Findings

• Latent communication beats text communication and single-agent baselines. With Qwen2.5-7B-Base on Alfworld, Interlat reaches 70.48% on seen and 65.42% on unseen tasks, versus 64.29% / 62.44% for the Text variant, 62.14% / 62.19% for No-Comm, 67.14% / 64.93% for CoT (full), and 65.71% / 62.69% for No-CoT. Gains hold for Qwen2.5-0.5B-Base (61.19% / 57.46% versus 54.52% / 47.26% for Text and 50.48% / 44.03% for No-Comm) and LLaMA3.1-8B-Base (70.71% / 70.90%).

• Agents given latents explore more, not less efficiently. Latent communication enables longer yet more successful trajectories, which the authors interpret as informed exploration rather than random wandering, and this pattern is analyzed in Appendix D.

• Cross-family communication produces the strongest gains. Feeding Qwen2.5-7B latents into a LLaMA3.1-8B actor yields 70.95% seen and 71.39% unseen, which the authors argue cannot be explained by architectural compatibility.

• Latents carry task-specific semantics. Replacing matched latents with cross-task latents drops performance substantially (CrossTask at 61.43% seen / 61.94% unseen for Qwen2.5-7B-Base); covariance-matched Gaussian surrogates and random orthogonal rotations degrade it further, as do additive and white-noise perturbations.

• A visible "aha" moment in training dynamics. The separation loss plateaus near ln 2 (0.69) for roughly the first 2k steps, then drops sharply after about 2.2k steps, marking when the actor begins exploiting task-relevant latents.

• Symbolic reasoning shows an inversion at high difficulty. On MATH, Interlat scores 36.88% overall, versus 38.35% for CoT (full), but on Level-5 tasks it leads with 15.80% versus 15.05% for CoT; the authors attribute this to natural language acting as a beneficial regularizer on simpler problems but causing "premature collapse" on hard ones.

• Aggressive compression works when trained. Compression training keeps success rates high and stable from 8 to 128 latent steps (roughly 1.8% to 28.8% of the full sequence); naive truncation peaks at 50% retention (72.14% seen / 61.19% unseen) then degrades. Trained latents reach 66.43% seen / 60.45% unseen at 8 steps, while untrained 8-step latents give 64.00% / 57.46%.

• Compression cuts latency dramatically. End-to-end message generation latency falls from 9.19 s to 0.39 s with 8-step latents (nearly 24× speed-up), and to 0.20 s with a lightweight bridge module.

• Compression preserves predictive confidence. The task-averaged relative change in cross-entropy decreases monotonically with the communication rate R and plateaus between roughly 30% and 75%; learned compression consistently yields lower cross-entropy than training-free truncation, with a maximum gap of about 11 percentage points. Trained latents show stable top-6 gaps and lower P50(S_10) than untrained latents, which collapse toward Top-1.

• Ablations show what matters most. For the actor, removing the communication adapter is catastrophic (4.05% seen / 4.48% unseen), removing curriculum learning collapses performance (33.10% / 20.65%), and removing the separation or alignment losses also hurts (58.81% seen for w/o L_sep; 56.90% for w/o L_align). For the reasoning model (at K = 128), the geometry loss is most critical (64.05% seen / 59.45% unseen), followed by the agreement loss (64.76% / 60.20%); removing the task loss gives 65.71% seen but 63.18% unseen.

• Latent communication scales to larger topologies. On ALFWorld with Qwen2.5-0.5B-Base, latent chain-3 (62.14% / 59.44%) and tree-3 (62.85% / 60.77%) beat their text counterparts (58.34% / 51.89% and 61.19% / 56.75%) and the 2-agent latent baseline (61.19% / 57.46%).

Methodology in Plain English

The authors set up two agents. A reasoning agent reads a task, produces a plan, and — at each decoding step — they save the model's last-layer hidden state immediately before it predicts that step's token. The resulting sequence of hidden states is the "thought" that gets sent over.

On the receiving side, the actor agent builds its input not from tokens but from a mix: the normal token embeddings of the prompt, then the incoming hidden states embedded directly into the sequence between <bop> and <eop> markers, then the rest of the prompt. Because the sender's hidden states live in a different representational space than the actor's embeddings, a small trainable communication adapter (self-attention plus a projection layer) rescales and interprets them.

Training the actor is not just next-token prediction. Alongside the standard task loss, the authors add a separation term that pushes apart the actor's output distributions when it sees latents from the right task versus latents from a different task in the same batch. They also add an alignment term that pulls the latent-conditioned distribution toward the distribution the same model would produce if it had been given the original text plan instead — this prevents the model from gaming the separation objective by drifting toward weird tokens. Because learning to read latents from scratch is unstable, training begins with a curriculum that randomly replaces a fraction r of the early communication positions with the corresponding plan token embeddings, sampled from {0, 0.1, ..., 1.0}.

For compression, they freeze the actor and adapter and train a separate reasoning model that generates a shorter latent message of K steps by feeding its own last hidden state back as the next input embedding through a small projection — a fully differentiable latent loop with no decoding to tokens. Three losses guide it: a task cross-entropy on the frozen actor's predictions, an agreement loss that aligns the actor's behavior under compressed versus full-length latents weighted by how much the latents reduce predictive uncertainty, and a geometry loss that keeps the step-averaged direction of the compressed latents aligned with the full-length ones.

Experiments use Alfworld with training data from Song et al. (2024) and MATH, with Qwen2.5-7B/0.5B-Base and LLaMA3.1-8B-Base as actors, instruction-tuned counterparts generating CoT plans and compression-free latents, and base models as reasoning models for compression. Alfworld episodes are capped at 20 steps. Training uses bfloat16, FlashAttention-2, and DeepSpeed, with AdamW at a learning rate of 1×10⁻⁵, global batch size 16, and 3% linear warmup; λ_task = 1, λ_sep annealed in [0.1, 1.0], λ_align in [0.1, 0.2]; compression training uses a learning rate of 5×10⁻⁵ and unit weights for all three objectives. Models are selected on a 5% validation split and all results are averaged over three independent runs.

Why This Matters

Impact on research. The work reframes inter-agent communication as a representation-transfer problem rather than a language-generation problem, and it also touches on single-model latent reasoning research (pause tokens, filler tokens, latent coprocessors, feedback of last hidden states). The paper notes the bandwidth argument explicitly: roughly 15 bits per token versus roughly 40k bits per hidden state. It also preserves agent autonomy by requiring no parameter sharing, memory coupling, or cache synchronization, which distinguishes it from prior hidden-state communication work such as activation grafting, latent deltas tied to language trajectories, and autoencoder-inferred thought prefixes.

Real-world applications (as implied by the setting, not measured in the paper):

• Multi-agent task automation in embodied or interactive environments, where a planner agent could hand rich guidance to an executor without verbose text.
• Latency- and cost-sensitive agent pipelines, where compressing communication to as few as 8 latent steps could cut message-generation time by nearly 24×.
• Mixed-vendor or mixed-model fleets, since the cross-family experiment suggests latents from one model family can train an actor from another.
• Parallel multi-agent verification setups, such as explorer-plus-critic configurations feeding a single actor, which the tree topology tests.

Industry relevance. The work comes from Zhejiang University's State Key Lab of CAD&CG, Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence, Shanghai Jiao Tong University, and the Future Living Lab of Alibaba, and the paper positions efficiency and cross-model compatibility as practical motivations. The limitation section is candid, though: the approach assumes access to last-layer hidden states, an assumption the authors state may not hold for closed-source or API-only models.

Future Directions

• Scaling beyond two agents and two benchmarks. The authors state that their setting is a controlled two-agent setup evaluated on one embodied interactive benchmark (ALFWorld) and one non-interactive symbolic benchmark (MATH), and call for extension to larger-scale, more heterogeneous agent ecosystems with dynamic role assignment and long-horizon collaboration.

• Supporting tool use, retrieval, and external memory. These components were deliberately excluded to reduce confounding factors, but the authors identify richer environments involving them as a natural extension.

• Resolving the in-distribution versus generalization trade-off. Removing the compression task loss degraded seen-task performance but slightly improved unseen-task performance (63.18% unseen), and the authors explicitly leave deeper investigation of this trade-off to future work.

• Working around the hidden-state access requirement. Since latent communication depends on internal representations, adapting the paradigm to closed-source or API-only models remains an open question per the limitations.

Target Audience

This paper is most useful for machine learning researchers and engineers working on multi-agent LLM systems, latent or continuous-space reasoning, and efficient agent communication. It also suits practitioners interested in reducing inter-agent latency and token cost, and readers following cross-model or heterogeneous-agent collaboration. Readers without background in transformer internals, supervised fine-tuning, or divergence-based objectives will find the methodology sections dense, though the motivation and main results are accessible.

Authors’ abstract

While natural language is the de facto communication medium for LLM-based agents, it presents a fundamental constraint. The process of downsampling rich, internal latent states into discrete tokens inherently limits the depth and nuance of information that can be transmitted, thereby hindering collaborative problem-solving. Inspired by telepathy, which bypasses symbolic language in communication, we propose Interlat (Inter-agent Latent Space Communication), a paradigm that leverages the continuous last hidden states of an LLM as a representation of its thought for direct communication (termed latent communication). An additional learned compression process further compresses latent communication via latent space reasoning. Experiments demonstrate that Interlat outperforms both fine-tuned chain-of-thought (CoT) prompting and single-agent baselines, even across heterogeneous models, promoting more exploratory behavior and enabling genuine utilization of latent information. Further compression not only substantially accelerates inference by up to 24 times but also maintains competitive performance through an efficient information-preserving mechanism. We position this work as a feasibility study of entirely latent space inter-agent communication, and our results highlight its potential, offering valuable insights for future research. Our code is available at https://github.com/XiaoDu-flying/Interlat.