Selective and Hybrid-State Communication for Latent Multi-Agent Systems

AI summary

Overview

Research area: Multi-agent large language model systems, specifically latent (internal-state) communication between agents, and inference-state management across Transformer and hybrid linear-attention architectures.

Technical level: Advanced. The paper assumes familiarity with KV caches, attention mechanics, and multi-agent orchestration, and includes formal notation for cache partitioning and tensor shapes.

Scope: Two independently evaluated methods — SelectKV (selective KV-state transfer for Transformer agents) and HyLaMAS (hybrid-state transfer for Qwen3.5-style models) — evaluated across six reasoning, knowledge, and code-generation benchmarks.

What This Paper Is About

Most LLM multi-agent systems talk to each other in natural language, which forces every agent to decode its internal state into text and the next agent to tokenize and re-encode it. Latent communication avoids that by passing internal model state directly, but it raises a new question: exactly what internal state should cross an agent boundary? The paper answers this in two ways — by sending only a selected subset of newly generated KV positions (SelectKV), and by extending latent handoff to hybrid architectures whose state includes recurrent linear-attention state in addition to KV caches (HyLaMAS).

Key Contributions

1. SelectKV — selective inter-agent KV communication. A boundary-aware mechanism for latent multi-agent handoff that protects KV positions inherited from earlier agents from repeated compression, while selecting from newly generated positions using a relevance score (cosine similarity between head/layer-averaged keys and a sender-derived query) and a local KV-novelty score.
2. HyLaMAS — hybrid-state latent communication. A shared in-memory latent workspace for hybrid linear-attention models that transfers full-attention KV caches together with the recurrent and causal-convolution states of Gated DeltaNet (GDN) layers, rather than projecting one KV cache into another.
3. Multi-benchmark evaluation. Six benchmarks (GSM8K, MedQA, ARC-Easy, ARC-Challenge, MBPP-Plus, HumanEval-Plus) with 100 examples each, covering math/science, commonsense reasoning, and code generation.
4. Ablations on boundary protection, selection criterion, and state channels. Matched-budget comparisons against random and recency-only selection, plus a channel ablation separating KV from GDN state.

Main Findings

• SelectKV cuts KV traffic modestly but consistently. Across all six benchmarks on Qwen3-4B with 10 latent steps, SelectKV-90 (90% retention of newly communicable positions) reduces communicated KV positions and logical KV payload by 9.7–9.9%.
• Accuracy is largely preserved, with one clear exception. Accuracy is unchanged on ARC-Easy (99.0 → 99.0) and ARC-Challenge (90.0 → 90.0), one point lower on GSM8K (94.0 → 93.0), MedQA (63.0 → 62.0), and MBPP+ (72.0 → 71.0), and six points lower on HumanEval+ (89.0 → 83.0) despite a 9.8% payload reduction — evidence that a fixed retention policy has task-dependent costs.
• Inference-time effects are mixed for SelectKV. Speed ratios versus uncompressed LatentMAS range from 1.10× (ARC-Easy, MBPP+), 1.09× (ARC-Challenge), 1.29× (MedQA), to slightly slower on GSM8K (0.97×) and HumanEval+ (0.99×). Output tokens fell on four benchmarks but rose 0.7% on GSM8K and 7.1% on HumanEval+.
• HyLaMAS reduces decoding work substantially. On Qwen3.5-4B and Qwen3.5-9B, HyLaMAS generates 53–74% fewer decoded output tokens than text-based multi-agent communication (TextMAS) and achieves 1.16–2.15× faster inference.
• The efficiency gain holds at 2B scale. On Qwen3.5-2B, HyLaMAS reduces output tokens by 74.1–80.0% with a 2.10–2.78× speedup, and accuracy is never lower than TextMAS, with differences from 0 to 13 percentage points — the largest gain being GSM8K (80% vs. 67%).
• HyLaMAS is not uniformly better than a single agent. On Qwen3.5-9B MBPP+, single-agent inference reaches 85% accuracy versus 76% for HyLaMAS, so the paper frames its results as trade-offs of replacing textual communication rather than blanket improvements.
• Boundary protection matters. In the MedQA ablation, boundary-aware SelectKV reaches 62% accuracy versus 58% for the same 90% retention without boundary protection, compared with 63% for full-cache LatentMAS, while payload drops from 295.81 MiB to 266.61 MiB per problem.
• The selection criterion beats matched-budget baselines. At an identical retained-KV budget on MedQA, SelectKV gets 62% accuracy versus 56% for random selection and 11% for recency-only selection.
• The hybrid channel ablation is inconclusive on accuracy. On Qwen3.5-9B MedQA, accuracy is 88% with both channels, 87% with KV only, 84% with GDN state only, and 90% with neither; none of these differences is significant under paired McNemar tests (p = 1.000, 0.344, and 0.754). A descriptive reduction in response length appears when GDN state is retained (825 vs. 983 decoded tokens with KV present; 724 vs. 979 without KV), and full-state transfer yields 825 vs. 979 tokens per completed response (15.7% fewer).

Methodology in Plain English

The researchers set up a four-agent chain — Planner, Critic, Refiner, Judger — where every agent shares one model instance and one set of weights, differing only in prompt and in the state they inherit. The first three agents do "latent rollouts": they keep computing internally without writing out any text. Only the Judger decodes a final answer.

For SelectKV, each handoff splits the growing KV cache into an inherited region and a newly generated region. The inherited region is left untouched so earlier agents' context is not repeatedly compressed. From the new region, positions are scored two ways: how similar their (averaged) keys are to a query vector derived from the sending agent's final hidden state, and how locally unusual they are compared with their neighbors. Positions ranking high on both are kept first, with a small set of recent positions protected. The retention ratio used is r = 0.90, chosen from preliminary experiments. Notably, the novelty score needs no attention-weight extraction.

For HyLaMAS, the observation is that Qwen3.5-style models interleave Gated DeltaNet linear-attention layers with full-attention layers, so a KV-only handoff loses information. HyLaMAS keeps a shared in-memory cache holding, for full-attention layers, the key and value tensors, and for GDN layers, both the recurrent matrix state and the recent projected inputs needed by the layer's causal convolution. Each agent continues computation from that shared state rather than reconstructing it from decoded text. Experiments use 40 latent steps per non-Judger agent, greedy decoding, batch size 8, and token caps of 2048 (ARC-Easy, ARC-Challenge, GSM8K) or 4096 (MedQA, MBPP+, HumanEval+).

Why This Matters

Impact on research. The paper reframes latent multi-agent communication as a state-selection and state-representation problem rather than solely a bandwidth problem, and it is one of the first to argue that KV-cache-centric handoff breaks down for hybrid architectures. The negative and neutral results — a six-point HumanEval+ drop, and a channel ablation where the no-transfer configuration scores highest — are useful counterweights to compression papers that report only favorable aggregates.

Real-world applications:

• Cost reduction in deployed multi-agent pipelines for code review or document analysis, where most of the expense is repeated decoding and re-encoding of intermediate text.
• Long-horizon agentic workflows where reasoning trajectories grow large and full-state transfer becomes the dominant cost.
• Serving systems built on hybrid linear-attention models, which need a defined handoff format for state that is not a KV cache.
• On-device or memory-constrained multi-agent setups, where the fixed-size recurrent GDN component is attractive relative to a cache that grows with sequence length.

Industry relevance. The measured wall-clock speedups (1.16–2.78×) and token reductions (53–80%) target inference cost directly. The paper is explicit, however, that logical payload reductions do not establish physical network savings or proportional end-to-end speedups: selection itself costs computation, and hybrid-state handoff requires state management.

Future Directions

• Scale up the evidence. The authors note that only 100 examples per benchmark and a single SelectKV model scale (Qwen3-4B) were evaluated; broader benchmarks, more model scales, and longer agent chains are needed to judge generalization.
• Combine the two methods. SelectKV and HyLaMAS were evaluated independently, not as a joint selective hybrid-state system — an obvious next step the paper names directly.
• Move from logical to physical transfer. Measuring distributed transfer, actual network savings, and adaptive communication policies is left to future work.
• Clarify the hybrid channel's role. The channel ablation showed no significant accuracy differences among KV-only, GDN-only, both, and neither, while response lengths differed descriptively; understanding when the recurrent state genuinely helps (as opposed to shortening outputs) remains open.

Target Audience

Researchers and engineers working on multi-agent LLM orchestration, KV-cache compression, or efficient inference serving. It is most useful to readers already comfortable with attention internals and cache management (advanced level), and to practitioners deciding whether selective or hybrid-state handoff is worth implementing in a production agent pipeline. Readers new to KV caches will find the notation in Section 3 heavy going, but the ablation tables are readable on their own.

Authors’ abstract

Efficient communication remains a major bottleneck in large language model multiagent systems. Latent communication reduces repeated natural-language exchange by transferring internal model states between agents, but these transfers remain costly and are typically designed around conventional KV-cache representations. We address this problem from two complementary perspectives. We introduce SelectKV, a boundary-aware method that selectively communicates KV states while protecting inherited context, and HyLaMAS, which extends latent collaboration to hybrid linear-attention models by communicating both KV caches and recurrent states through a shared workspace. Across six benchmarks, SelectKV reduces communicated KV positions and logical KV payload by 9.7-9.9%, with accuracy unchanged on two benchmarks, within one percentage point on three, and six points lower on HumanEval+. Across Qwen3.5 model scales, HyLaMAS reduces decoded output tokens by 53-80% and achieves 1.16-2.78x speedups relative to text-based multi-agent communication. Together, these independently evaluated methods represent two complementary approaches to efficient latent-state communication across conventional Transformer and hybrid model architectures.