MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks

AI summary

Overview

• Research area: Multi-agent systems (MAS) built on large language models, specifically automatic MAS design via reinforcement learning, plus benchmark design for controlled MAS-vs-SAS comparison.
• Technical level: Advanced. The paper uses formal RL notation (policy π_θ, GRPO group-relative objectives), function-calling abstractions, and a benchmark construction pipeline that assumes familiarity with agent orchestration concepts.
• Scope (one sentence): The paper proposes a training-time framework that generates an entire multi-agent system in a single decision step, and pairs it with a controlled five-axis benchmark for determining when multi-agent coordination actually beats a single agent.

What This Paper Is About

Current automatic MAS design under-delivers for two reasons the authors identify: orchestration is typically expressed as sequential, code-level execution that limits global system-level reasoning and scales poorly as sub-agents grow complex, and MAS are deployed without evidence that they outperform simpler single-agent systems (SAS). The paper's goal is to fix both problems — by reformulating orchestration as a holistic, function-calling reinforcement learning problem, and by building a benchmark that can say, in a controlled way, when multi-agent coordination is worth its cost.

Key Contributions

1. A holistic, function-calling RL formulation for MAS orchestration. Complex, goal-oriented sub-agents are encapsulated as callable functions, exposing only their signature to the orchestrator. Two primitive functions — create_agent and create_flow — let the orchestrator generate the complete MAS in a single decision step. An explicit user-configurable "degree of MAS" (DoM) variable, set to Low or High, constrains the allowable orchestration space (under Low DoM, at most one sub-agent and no explicit inter-agent topology).

2. MasBench, a controlled benchmark with five evaluation axes. The axes are Depth, Horizon, Breadth, Parallel, and Robustness, derived from the structural properties of each instance's dependency graph plus its verification protocol. The paper states this is the first benchmark established to evaluate MAS benefits.

3. A systematic empirical analysis of MAS versus SAS. Three analysis directions covering 3 orchestrator settings and 5 sub-agent settings across different model sizes and families, on a broad range of MAS configurations.

4. Consistent gains on public benchmarks with efficiency. MAS-Orchestra improves on math reasoning, multi-hop QA, and multi-step search-based QA, while achieving more than 10× efficiency over strong baselines and sitting on the performance–cost Pareto frontier.

Main Findings

• MAS benefits depend on task structure, not universally. With a "weaker" sub-agent (Qwen-7B-Instr), MAS (MAS-Orchestra) outperforms SAS across most sub-task structures except the Depth axis, where strongly interdependent sub-tasks in strict sequential order favor a single sequential chain-of-thought with less branching and coordination overhead.

• Stronger sub-agents shrink the MAS advantage. With a "stronger" sub-agent (GPT-OSS-120B at low reasoning effort), gains diminish across Depth, Horizon, Breadth, and Parallel; coordination cost and error propagation across agents can offset potential gains.

• Robustness is where MAS wins decisively. MAS consistently show superior Robustness under data poisoning, while SAS performance collapses to near-zero accuracy. The paper attributes this to redundancy, cross-verification, and structured inter-agent interactions; in one example, MAS introduces a final answer agent acting as a moderator that identifies and mitigates adversarial signals.

• MAS are most effective at the edge of sub-agent competence. Gains are clear when the underlying sub-agent is capable but not yet strong enough to reliably internalize complex task structure on its own.

• Instruction-tuned LLM orchestrators beat reasoning LLM (RLM) orchestrators. RLM-based orchestrators tend to produce much simpler MAS — at times only one sub-agent, which becomes the dominant pattern as training converges — and tend to solve the task itself first and then delegate to one simple sub-agent even when the sub-agent is stronger. Instruction-tuned orchestrators are more flexible in delegation, with a three-agent case becoming dominant.

• Higher sub-agent reasoning effort does not improve accuracy. Increasing reasoning effort does not beat low reasoning effort for either MAS or SAS, because higher effort substantially increases the likelihood of exceeding the default maximum context length. Robustness gains for MAS persist when maximum context is raised to its limit (120k tokens for the GPT-120B sub-agent).

• Learned orchestration adapts to the task. Under Low DoM, MAS-Orchestra delegates 100% of tasks to a single sub-agent after 20 steps of training, primarily selecting ReflexionAgent and DebateAgent, the best-performing SAS baselines. Under High DoM, it exploits parallelism by invoking DeepResearchAgent for multiple parallel searches, typically 3 to 4 per question.

• Public benchmark results (Avg@8 accuracy). MAS-Orchestra scores 66.25 on AIME24, 61.25 on AIME25, 49.00 on HotpotQA, 11.00 on BrowseComp+, and 65.21 on GPQA, versus the best standalone agent baselines of 62.08 (DebateAgent, AIME24), 57.50 (DebateAgent, AIME25), 46.44 (DeepResearchAgent, HotpotQA), 8.56 (DeepResearchAgent, BrowseComp+), and 64.14 (DebateAgent, GPQA). Among SoTA inference-time orchestration systems, AFlow scores 62.50 / 53.33 / 65.43 (AIME24 / AIME25 / GPQA) and MaAS scores 32.50 / 40.83 / 40.78; MAS-Zero generated no valid MAS with a 7B orchestrator. Among public training-time orchestration systems, MAS-GPT scores 58.75 / 43.33 / 63.51 and ToolOrchestra scores 23.33 / 11.25 / 37.44 / 1.38 / 29.80.

• OOD generalization holds. Because DeepScaleR training data centers on math reasoning and is largely out-of-domain for GPQA, the GPQA result (65.21) serves as evidence that the learned orchestration transfers rather than overfitting to training distributions.

• DoM should be chosen per task. The authors use Low DoM for the largely sequential AIME and GPQA tasks, and High DoM plus a DeepResearchAgent for HotpotQA and BrowseComp+, which involve complex sub-task structures such as parallel search.

Methodology in Plain English

The researchers start from a design belief: an orchestrator should do high-level planning, not reproduce the internals of its sub-agents. So they define each sub-agent as a goal-oriented black box with a name and configurable parameters. The orchestrator never sees how a sub-agent reasons; it only emits function calls that say which agents to create and how they connect. A deterministic rule-based parser then reads that specification, instantiates the agents with their assigned goals, runs them, and returns the final prediction.

Training treats this as a reinforcement learning problem over the orchestrator's policy. For a given question and a chosen DoM level, the policy samples an orchestration; a reward is assigned based on final answer correctness (and, for the Horizon axis, intermediate answer correctness). Optimization uses Group Relative Policy Optimization (GRPO): for each input the system samples a group of K orchestrations, computes their rewards, and updates the policy by comparing each sample against the others in its group. Because the whole MAS is emitted in one step, the orchestrator never observes intermediate execution states — credit is assigned purely from the end result, which the authors argue avoids long-horizon credit assignment problems and error propagation found in sequential, multi-turn orchestration training.

To study when MAS helps, they build MasBench from the synthetic data generator iGSM, which allows controllable dependency graphs and automatic natural-language question derivation. Depth, Horizon, Breadth, and Parallel come straight from graph properties (longest dependency chain containing the answer, number of intermediate sub-tasks whose answers must be carried forward, maximum in-degree, and number of independent sub-task components). Robustness is created by augmenting each sub-task with a short adversarial note containing an incorrect answer from an upstream sub-task; in practice this is applied to Depth set to 4, by interleaving iGSM subtask instructions with needle-in-a-haystack information extraction from the RULER benchmark. Splits are kept non-overlapping at template level by using different hash values during generation.

In the controlled analyses, they fix all sub-agents and vary one factor at a time — only CoTAgent as a candidate sub-agent to remove confounding from heterogeneous capabilities, with SAS using the same CoTAgent untrained, and DoM set to High. They then apply the insights to five public benchmarks: AIME24, AIME25, GPQA, HotpotQA, and BrowseComp+, using DeepScaleR as training data for AIME and GPQA, the provided HotpotQA training split, and 80% of BrowseComp+ for training the orchestrator. The orchestrator is Qwen-7B and the sub-agent backbone is GPT-OSS-120B at low reasoning effort. Candidate sub-agents share the same LLM backbone and differ only in tools and prompting workflows: CoT, CoT with self-consistency, Debate, Self-refine, and a Search agent that decides when and what to search in multi-turn fashion following OpenDeepResearch.

Why This Matters

The work reframes multi-agent design from "build a pipeline" to "train a planner," and supplies the missing measurement apparatus for asking whether coordination is justified at all. Its comparison table positions MAS-Orchestra as the only method among the listed systems (MAS-Zero, AFlow, DyLAN, MAS-GPT, CoRL, Puppet, xRouter, W4S, ToolOrchestra) supporting goal-oriented sub-agents, holistic orchestration, configurable DoM, and controlled MAS-vs-SAS analysis simultaneously.

• Real-world application — math and science tutoring/QA. The AIME24, AIME25, and GPQA results show a single trained orchestrator can adapt between delegation and direct solving, which matters for systems that must handle both routine and stepped-up difficulty without a human choosing the architecture.
• Real-world application — research assistants and evidence aggregation. The HotpotQA result (49.00 versus a 46.44 best single agent) reflects multi-hop evidence gathering, a pattern common in literature review and analyst tooling.
• Real-world application — search-heavy agents. The BrowseComp+ result (11.00 versus 8.56 for DeepResearchAgent alone) targets deep, multi-turn retrieval tasks where parallel search plus aggregation matters.
• Real-world application — adversarial or noisy data pipelines. The Robustness axis speaks to retrieval systems ingesting poisoned or misleading content; MAS's near-zero-collapse resistance contrasts sharply with SAS accepting the adversarial note directly.
• Industry relevance. Efficiency is framed as a first-class outcome: more than 10× efficiency over strong baselines and placement on the performance–cost Pareto frontier on AIME24 and GPQA matter for serving cost. The explicit DoM knob gives practitioners a way to encode the finding that MAS should be deployed selectively rather than as a default replacement for single-agent systems. Code, the MasBench dataset, a project page, and a live demo are released.

Future Directions

• Context management and length control for higher reasoning effort. Because performance degrades once the context limit is hit, the authors conclude that handling longer reasoning traces requires explicit training for context management and length control.
• Extending robustness evaluation beyond one axis value. Robustness is applicable to all structural task types but is applied only to Depth set to 4; whether the resilience pattern generalizes across the other structural settings remains open.
• Accounting for ensembling effects. The paper explicitly notes that MAS gains may stem from effective coordination and specialization or from increased effective compute via ensembling, and that separating these is unresolved.
• Better use of reasoning LLMs as orchestrators. RLM-based orchestrators prioritized direct solving over delegation and produced simpler MAS; whether RLMs can be trained or prompted to orchestrate well is left open.

Target Audience

Researchers and engineers working on LLM agents, agent orchestration, and reinforcement-learning-based system design, particularly those deciding whether to build multi-agent pipelines or a single strong agent. It is also relevant to benchmark designers and to practitioners responsible for cost, latency, and reliability of deployed agentic products. Readers need comfort with RL terminology (policies, rewards, group-relative updates) and with agent concepts such as tools, context, and inter-agent topology; the benchmark's five axes are explained concretely enough to be usable by those who skip the formal sections.

Authors’ abstract

While multi-agent systems (MAS) promise elevated intelligence through coordination of agents, current approaches to automatic MAS design under-deliver. Such shortcomings stem from two key factors: (1) methodological complexity - agent orchestration is performed using sequential, code-level execution that limits global system-level holistic reasoning and scales poorly with agent complexity - and (2) efficacy uncertainty - MAS are deployed without understanding if there are tangible benefits compared to single-agent systems (SAS). We propose MASOrchestra, a training-time framework that formulates MAS orchestration as a function-calling reinforcement learning problem with holistic orchestration, generating an entire MAS at once. In MAS-Orchestra, complex, goal-oriented subagents are abstracted as callable functions, enabling global reasoning over system structure while hiding internal execution details. To rigorously study when and why MAS are beneficial, we introduce MASBENCH, a controlled benchmark that characterizes tasks along five axes: Depth, Horizon, Breadth, Parallel, and Robustness. Our analysis reveals that MAS gains depend critically on task structure, verification protocols, and the capabilities of both orchestrator and subagents, rather than holding universally. Guided by these insights, MAS-Orchestra achieves consistent improvements on public benchmarks including mathematical reasoning, multi-hop QA, and search-based QA, while achieving more than 10x efficiency over strong baselines. Together, MAS-Orchestra and MASBENCH enable better training and understanding of MAS in the pursuit of multi-agent intelligence.