Multi-Agent Relay Thinking

AI summary

Overview

Research area: Multi-agent collaboration among multimodal large language models (MLLMs) for visual reasoning — the paper sits at the intersection of multi-agent systems, uncertainty/confidence estimation, and multimodal reasoning benchmarks.

Technical level: Intermediate. Readers should be comfortable with autoregressive language-model generation, token log-probabilities, and the general vocabulary of multi-agent prompting; the confidence calibration step involves a standard ANOVA and z-score normalization but no advanced mathematics.

Scope: The paper proposes and evaluates Macr (Multi-Agent Confidence-guided Relay), a framework that replaces parallel "group thinking" multi-agent collaboration with a sequential "relay thinking" topology in which calibrated model confidence decides when reasoning ownership is handed off and which of three heterogeneous MLLMs takes over.

What This Paper Is About

Most multi-agent collaboration methods have agents solve a problem independently and then merge their answers — the authors call this group thinking. This paper instead explores relay thinking, where one model starts reasoning, hands the partial reasoning trajectory to a second model when it gets stuck, and that model continues from where the first left off. The core difficulty is deciding when to hand off and to whom, which the authors solve by making each model's confidence comparable across different architectures and using that calibrated confidence as the trigger and the selector.

Key Contributions

1. A new collaboration topology. The paper reframes multi-agent collaboration around relay thinking, drawn from cognitive science, where reasoning ownership is dynamically transferred across agents so that a single evolving reasoning trajectory is progressively refined rather than aggregated after the fact.

2. The Macr framework. A confidence-guided relay system that runs an online "initiation–monitoring–bidding–relay" loop: it picks a starting model by confidence, monitors the leading model's local confidence during generation, interrupts when confidence falls below a threshold, and runs a short bidding round in which every model proposes a continuation.

3. A calibration method for cross-model confidence. The authors show that raw confidence from heterogeneous models is not comparable (ANOVA F = 13.31) and introduce a sliding-window standardization using each model's historical mean and standard deviation, which reduces the F-value to 1.01.

4. Two empirical findings about heterogeneous collaboration. Models show complementary strengths at different reasoning stages, and reasoning bottlenecks appear at different stages for different tasks — motivating dynamic scheduling of both ownership and relay timing.

Main Findings

• Accuracy gains over the strongest single model: Macr reaches 78.00% on MathVista versus 74.70% for the best single-model strategy (+3.30%), and 72.00% on MMStar versus 68.13% (+3.87%). The abstract rounds these to +3.3% and +3.9%.

• Comparable to multi-agent baselines at far lower token cost: H-DMAD scores 78.80 on MathVista, 73.20 on MMStar, and 75.50 on HallusionBench versus Macr's 78.00, 72.00, and 74.66 — i.e., slightly lower accuracy on those three — while Macr uses 6.00× fewer tokens (4,414 vs 26,375 on HallusionBench, and 4,150 vs 24,677 on MMStar). Macr does edge out H-DMAD on LogicVista (55.80 vs 55.58).

• Best efficiency (IBC): Macr leads on Incremental Benefit per Cost across all four datasets, with 21.35 on MMStar versus 2.83 for H-DMAD and 15.38 on MathVista versus 2.63 for H-DMAD.

• Calibration works: Sliding-window local confidence (window size w = 16) shows clear distribution shifts across Qwen3 VL, InternVL 3.5, and MiniCPM-V 4.5 on 1,000 sampled MMMU instances; standardization aligns the curves around 0 and drops the ANOVA F from 13.31 to 1.01. Correct trajectories also show higher standardized confidence than incorrect ones.

• No single model dominates: In the row-normalized transition matrix, MiniCPM-V 4.5 has the highest self-transition rate at 59.7% but still hands off in 40.3% of bids; InternVL 3.5 transfers to Qwen3 VL 22.4% of the time and Qwen3 VL returns control 26.9% of the time.

• Models excel at different stages: A Kaplan-Meier survival analysis shows MiniCPM-V 4.5 with the highest early survival (number at risk 3,151, roughly twice the other two at 1,163 and 1,658), InternVL 3.5 holding a later-stage plateau, and Qwen3-VL having the poorest long-horizon retention.

• Relay timing is task-dependent: Logic-heavy tasks (LogicVista, MathVista) cluster relays early, while visual-intensive tasks (MMStar, HallusionBench) cluster them later; bidding frequency stabilizes in the final stage across all tasks.

• Every component matters (ablation on 500 MMStar instances): Macr scores 72.40% with 312 tokens. Fixed initiation drops to 71.60% (MiniCPM-V 4.5), 71.20% (InternVL 3.5), and 70.40% (Qwen3 VL); fixed selection order drops to 69.00%; fixed sentence-boundary triggering drops to 69.80%.

• Heterogeneity is required: With a single homogeneous model, role-playing (RPH) yields 68.00%, 70.20%, and 64.80%, and temperature-decoded sampling (TDH) yields 66.60%, 68.80%, and 63.60% — all below Macr's 72.40%.

• Bottlenecks have lexical signatures: Models hesitate on different vocabulary — InternVL on analytical verbs (analyze, examine, identify), MiniCPM-V on spatial/attribute terms (position, top, left, large), Qwen3-VL on causal connectives (since, therefore, suggesting) — but all frequently emit stall tokens such as wait, look, check.

Methodology in Plain English

Step 1 — Measure confidence locally, not per answer. Instead of averaging a model's probability over a whole response, the authors average log-probabilities over a small sliding window of recent tokens. This gives a running sense of how comfortable the model is right now, which is what you need to catch a mid-reasoning stall.

Step 2 — Make confidences comparable. Three different MLLMs produce probability scores on different scales, so their numbers cannot be compared directly. The authors collect local-confidence samples from each model on a reference dataset, compute that model's own mean and standard deviation, and z-score every new confidence value against them. After this, the three models' distributions line up and the ANOVA F-statistic falls from 13.31 to 1.01.

Step 3 — Pick a starter. All three models each generate a short w = 16 token opening from the same question; the one with the highest calibrated confidence becomes the initial leading model.

Step 4 — Watch for trouble. The leader generates normally while its calibrated local confidence is tracked. If it falls below a threshold (τ = −0.3, tuned on 600 RealWorldQA instances held out from all evaluation sets), generation halts.

Step 5 — Run a bidding round. The shared context is passed to every model, each produces a short candidate continuation, and the model with the highest calibrated confidence on its own candidate wins.

Step 6 — Relay. The winner's candidate tokens are appended to the context and that model takes over generation. Each model keeps its own KV cache, so the shared context does not need to be re-prefilled. The loop repeats until the answer is finished.

Evaluation. Three 8B-scale models form the pool: Qwen3-VL-8B-Instruct, InternVL3.5-8B, and MiniCPM-V-4.5-8B. They are tested on four benchmarks — MathVista (1,000), LogicVista (448), MMStar (1,500), and HallusionBench (951) — against single-model baselines (IO, CCoT, DDCoT) and multi-agent baselines (Majority Vote and H-DMAD), with three random seeds and reported mean ± standard deviation, plus token counts and IBC.

Why This Matters

Impact on research. The paper challenges the default assumption that multi-agent systems should generate in parallel and aggregate at the end. By showing that sequential handoff with calibrated confidence can match the accuracy of debate-style aggregation at a fraction of the token cost, it opens a scheduling-oriented research direction: treating a pool of models as a resource to be routed through during inference rather than as voters to be polled after it. The observation that confidence from different model families is not directly comparable — and that a simple historical standardization fixes it — is a reusable finding for any system that mixes heterogeneous models.

Real-world applications:

• Visual math and geometry tutoring, where a system can hand off to a different model when the current one stalls on a chart or a word problem.
• Chart, table, and document question answering, where heterogeneous model strengths could be combined without paying the cost of three full generations.
• Hallucination-sensitive visual verification (the HallusionBench setting), where catching a low-confidence moment mid-answer is more useful than checking a finished answer.
• Cost-constrained assistant deployment, where token budget — not just accuracy — determines what is feasible at scale.

Industry relevance. Token overhead is a direct operating cost, so a method that matches multi-agent accuracy at 6.00× fewer tokens on some benchmarks is immediately relevant to teams deploying ensembles of models. The per-model KV-cache design (no repeated prefilling) also speaks to serving efficiency, and the framework is model-agnostic within the tested pool.

Future Directions

• Tuning and generalizing the hyperparameters. The window size (w = 16) and threshold (τ = −0.3) were selected via parameter search on 600 RealWorldQA instances. Whether these transfer to other model pools, modalities, or task distributions — or whether they must be re-tuned each time — is not established.

• Scaling the pool. The study uses exactly three 8B-scale MLLMs. How bidding cost, calibration stability, and relay quality behave with larger or more heterogeneous pools is an open question, and bidding itself costs tokens (the reported totals include them).

• Beyond token counts. Latency, throughput, and memory are not reported; token overhead is the only efficiency metric, so real serving cost of running three models with three KV caches is unmeasured.

• Confidence as a scheduling signal beyond handoff. The authors suggest their findings should motivate dynamic model scheduling that adapts both reasoning ownership and relay timing throughout inference — including whether the same calibrated signal could drive other interventions, and whether lexical stall cues (e.g., wait, look, check) or stage-specific hesitation patterns could sharpen bottleneck detection.

Target Audience

Researchers and practitioners working on multi-agent LLM/MLLM systems, inference-time compute allocation, and uncertainty-aware reasoning will get the most from this paper. It is also useful for engineers building model-routing or ensemble-serving infrastructure who care about accuracy-per-token, and for graduate students looking for a clear example of how a cognitive-science framing (relay versus group thinking) can be operationalized into a concrete algorithmic loop with ablations for each component. Readers unfamiliar with token log-probabilities or ANOVA will need some background, which is why the paper reads as intermediate rather than beginner-level.

Authors’ abstract

Existing multi-agent collaboration (MAC) methods have primarily explored a group thinking collaboration topology, where agents independently generate solutions and aggregate their outputs. In this work, we investigate an alternative relay thinking topology, where reasoning ownership is dynamically transferred across agents, allowing subsequent agents to inherit and refine an evolving reasoning trajectory. We introduce Multi-Agent Confidence-guided Relay (MACR), a framework that enables dynamical relay collaboration among agents. The key challenge is determining when reasoning should be relayed and which agent should continue the process. method addresses this challenge through confidence-guided relay triggering and a bidding mechanism that selects the next agent based on its confidence. Experiments on four benchmarks show that method outperforms the strongest single-model strategies in accuracy, e.g., +3.3% on MathVista and +3.9% on MMStar, and remains on par with multi-agent strategies while consuming 6.0x less token overhead on HallusionBench and MMStar. Further analysis demonstrates that relay collaboration enables heterogeneous agents to progressively integrate complementary insights, combining heterogeneous capabilities across reasoning stages. Our source code is available at https://github.com/67L1/MACR.