DynaFlow: Learning Free-Form Multi-Agent Orchestration

AI summary

Overview

Research area: Multi-agent systems built on large language models, specifically the problem of orchestration — deciding when agents should collaborate and how their work should be structured and connected.

Technical level: Advanced. The paper assumes familiarity with supervised fine-tuning, LoRA/QLoRA, agent tool use, and standard LLM benchmarks.

Scope: DynaFlow is a framework that makes "write a multi-agent workflow program" a native action of a single task-solving agent, which is trained (while all sub-agents stay frozen) to decide when to collaborate and how to wire sub-agents together so that intermediate results flow correctly.

What This Paper Is About

Most multi-agent LLM systems use a fixed collaboration structure — pre-designed roles, communication patterns, or workflow templates — which does not fit tasks of varying difficulty and can add unnecessary communication, latency, and computation. The authors argue that scalable systems need free orchestration: the ability to decide during problem solving whether collaboration is useful and to construct a task-specific workflow on the fly. DynaFlow gives the main agent a JavaScript-based workflow action, executes it with frozen sub-agents, and trains only the main agent to use execution results and traces to guide its next move.

Key Contributions

1. Workflow synthesis as an action. The main agent's action space is extended to three options — ordinary tool calls, a final answer, or a single action (P, W(P)) pairing a synthesized JavaScript program P with its invocation through a workflow tool W — so collaboration decisions and workflow construction are part of one policy.
2. Two-stage training of the main agent only. Stage 1 distills 2,031 first-turn demonstrations from a GLM-5.3 Max teacher (§4; Appendix C reports 2,257 first-turn demonstrations). Stage 2 continues on 712 curated examples selected by an LLM judge, with all sub-agents held fixed at the Base Qwen3.5-27B checkpoint.
3. Orchestration-quality data screening. A judge (deepseek-flash) flags three defect types — premise injection, verification defects, and information-flow defects — with supporting excerpts required before a demonstration is kept.
4. An empirical account of what changes with training. Beyond accuracy, the paper analyzes six structural properties of generated workflows (execution structure, role specialization, dependency pattern, output reuse, aggregation strategy, agent count) and four design criteria, identifying cross-stage information flow as a key dimension of orchestration quality.

Main Findings

• Headline accuracy: DynaFlow reaches 88.08% overall accuracy in Forced mode and 87.94% in Dynamic mode, versus 77.18% for the strongest evaluated baseline (Self-Refine, same Qwen3.5-27B backbone) — a 10.90 percentage-point gap.
• Mathematics: Forced mode solves all 90 AIME questions (100% on AIME 2024, 2025, and 2026) and reaches 90.50% on PolyMATH; both modes beat the strongest baseline on PolyMATH by 3–4 points.
• Science: Forced mode reaches 84.85% on GPQA-Diamond, 7.07 points above Base, Forced — but Self-Refine (88.38%) and AFlow (87.37%) remain ahead on this benchmark.
• Retrieval: HotpotQA reaches 84.50% (Dynamic) and 83.50% (Forced), 6–7 points above Base + Web search (77.50%).
• Workflows help only after training: Without training, Forced mode lowers accuracy relative to Base, Direct on every benchmark except HotpotQA (Base, Forced = 75.87% vs Base, Direct = 71.51%). After Stage 1, Forced reaches 85.47%, exceeding Direct within the same checkpoint by 8.73 points.
• Training stages are additive: Base → Stage 1 → DynaFlow gives 71.51 → 76.74 → 77.18 (Direct), 65.12 → 84.45 → 87.94 (Dynamic), and 75.87 → 85.47 → 88.08 (Forced).
• The routing gap collapses: The Dynamic–Forced accuracy gap narrows from 10.75 points (Base) to 1.02 (Stage 1) to 0.14 (DynaFlow).
• Fewer agents, similar accuracy: In Dynamic mode DynaFlow answers 47.1% of questions without any workflow and uses 2.44 sub-agents per question versus about 4.16 under Forced execution, a reduction of roughly 41%. (Figure 1 reports a mean of 3.44 agent roles per question for Dynamic and 87.9% accuracy.)
• Structured, connected workflows: The share of hybrid (parallel plus sequential) workflows rises from 3.4% (Base) to 83.2% (Stage 1) to 85.6% (Stage 2); programs that explicitly require downstream agents to use upstream outputs rise from 12.6% to 93.3% to 97.7%.
• Curation reduces design defects: Major information-flow defects fall from 1.9% (Stage 1) to 0.9% (Stage 2), and workflows specifying correction or refinement of intermediate results rise from 28.1% to 35.9%, with fewer sub-agents on average (4.16 vs 4.35).
• Task-specific role allocation: Solvers and verifiers exceed 90% of nodes on AIME and PolyMATH; researchers are 59.6% of nodes on HotpotQA; verification accounts for 33.2–43.9% of nodes across tasks.
• Failure modes remain: On GPQA-Diamond, incorrect intermediate conclusions from sub-agents can misguide later verification, and the main agent can fail to prioritize correct evidence when synthesizing the final answer.

Methodology in Plain English

The main agent is given a small programming interface: it can call sub-agents with instructions, run calls in parallel, chain dependent stages, and attach phase labels. It writes a JavaScript program describing who does what and what information is passed where; a budget-constrained runtime executes that program with frozen sub-agents and returns both the output and a trace of each sub-agent call. The main agent then decides whether to answer, use an ordinary tool, or write another workflow.

Training targets only the first turn: the main agent's reasoning r plus its workflow action a₁. Stage 1 uses trajectories from a stronger teacher model, keeping only runs whose final answers are correct and which actually delegate the substantive solving to sub-agents. Stage 2 re-trains on a filtered subset, where an LLM judge inspects each program and its launch arguments for three failure types — claims asserted as established facts beyond the task givens, verification instructions that steer reviewers toward an expected answer, and broken information flow (missing context, wrong references, lossy transfer, hard-coded substitutes for upstream outputs). Every flagged defect must be backed by a quoted excerpt. A random-subset continuation control is used to separate the effect of curation from the effect of extra training.

Evaluation uses three modes: Direct (no tools or workflows), Dynamic (choose direct answer or workflow on the first decision), and Forced (a workflow must be invoked before substantive solving). All systems share the same Qwen3.5-27B backbone, and in workflow settings the main agent is restricted to designing, coordinating, and summarizing, with substantive solving delegated. Accuracy counts invalid final answers as incorrect. Training used 4-bit NF4 QLoRA on two NVIDIA H100 80GB GPUs, for about 52 GPU-hours (Stage 1) and 17 GPU-hours (Stage 2), ~69 GPU-hours total excluding teacher generation, evaluation, and judging.

Why This Matters

The paper shifts attention from "design a better fixed team" to "learn when a team is needed and how to wire it," and it isolates orchestration from executor quality by never training the sub-agents. The result that untrained workflow use hurts accuracy on five of six benchmarks while trained use helps on all six is a concrete warning for practitioners who assume adding agents is free.

Real-world applications:

• Enterprise research assistants that spawn retrieval and cross-checking agents only for questions needing multi-hop evidence, and answer simple questions directly.
• Scientific and technical QA with verification stages, where intermediate derivations are checked before being trusted.
• Multilingual and competition math tutoring, where symbolic derivation is paired with small-instance numerical checks.
• Cost- and latency-sensitive agent deployments, where routing ~47% of queries to a direct answer reduces sub-agent calls by about 41% with roughly unchanged accuracy.

Industry relevance: The main lever is a trainable policy over an existing tool interface, paired with a data-screening rubric that can be reused to audit any multi-agent pipeline. The GPU budget reported (~69 GPU-hours on two H100s) and the frozen-executor design make the approach compatible with reusing already-deployed models.

Future Directions

• Train sub-agents too. The authors suggest training executors for reliable information transfer and independent verification, rather than improving orchestration alone.
• Extend beyond first-turn supervision. Supervision currently covers only the first reasoning step and workflow action; multi-turn trajectories containing disagreements and erroneous intermediate results could teach error recovery, workflow revision, and better final-answer selection.
• Close the science gap. DynaFlow still trails Self-Refine (88.38%) and AFlow (87.37%) on GPQA-Diamond, and the failure analysis points to verification being misled by wrong intermediate conclusions.
• Generalize the evidence. Results come from one backbone family (Qwen3.5-27B) and six benchmarks; whether the curation rubric and gains transfer to other model scales and task families is not reported.

Target Audience

Researchers and engineers working on LLM agent systems, multi-agent orchestration, and agentic workflow synthesis, as well as practitioners who need to decide when multi-agent collaboration is worth its cost. The paper is most useful to readers comfortable with supervised fine-tuning and agent tooling; the workflow-quality rubric in Appendix E and the structural definitions in Table 5 are directly reusable for auditing existing pipelines.

Authors’ abstract

Multi-agent systems often rely on predefined collaboration structures, limiting their ability to adapt to diverse task requirements. We introduce DynaFlow, a framework that integrates free-form executable workflow synthesis into the action space of the main agent. The agent decides when collaboration is needed and generates task-specific programs that coordinate sub-agents through sequential or parallel execution, structured information exchange, and verification. Execution results and traces inform subsequent actions, enabling the agent to adapt collaboration as reasoning progresses. We train only the main agent through supervised fine-tuning on distilled trajectories, followed by refinement on examples filtered for unsupported premises, biased verification, and information-flow defects. All sub-agents remain fixed, separating orchestration learning from executor capability improvement. Across six benchmarks spanning mathematical reasoning, scientific question answering, and multi-hop retrieval, DynaFlow achieves 88.08% overall accuracy in Forced mode, outperforming the strongest evaluated baseline by 10.90 percentage points with the same Qwen3.5-27B backbone. Workflow analysis finds more explicit use, verification, and refinement of upstream outputs after training, highlighting cross-stage information flow as a key dimension of multi-agent orchestration.