CEPO: A General Framework for Improving Prompt Optimization in Multi-Agent Systems

AI summary

Overview

Research area: Large language model (LLM) multi-agent systems and automatic prompt optimization.

Technical level: Intermediate. The framework is conceptual (prompt structure plus optimization guidance) rather than mathematically heavy, but readers benefit from familiarity with prompt optimizers such as OPRO, ProTeGi, PE2, and with agent pipelines for document QA, math, code, and video.

Scope: The paper proposes CEPO, a framework that restructures how multi-agent prompts are parameterized and how they are updated, and shows across 11 benchmarks in four domains that it improves four different existing prompt optimizers.

What This Paper Is About

Multi-agent systems need two different things to work well: agents must coordinate as a team, and each agent must execute its own specialized job. In current prompt optimization methods, both functions are tangled inside each individual agent's prompt, so revising a shared coordination rule requires making consistent edits across many separate prompts with different responsibilities. The paper asks how prompt structure and optimization guidance can make existing prompt optimizers more effective for multi-agent systems, and answers with a design that splits a shared coordinator prompt from agent-specific executor prompts and gives each layer its own role-aligned optimization instructions.

Key Contributions

1. Two design principles for decoupling coordination from execution. The paper identifies the coupling of system-level coordination and agent-level execution as a common limitation of existing multi-agent prompt optimization, and proposes (a) coordinator-executor prompt decomposition and (b) role-aligned optimization guidance.

2. CEPO, a general framework rather than a new optimizer. CEPO specifies only the prompt structure and the meta-instructions; the actual candidate generation and search are carried out by whatever underlying optimizer the user supplies. The paper instantiates it on four: PE2, OPRO, ProTeGi, and MASPO.

3. Broad empirical validation. Applying CEPO to those four optimizers yields consistent improvements across 11 benchmarks spanning document understanding, mathematical reasoning, code generation, and video understanding.

4. Analysis of design choices and system behavior. Component ablations, a meta-instruction swap study, an optimization-schedule ablation, a target-executor-set ablation, and controlled studies on a document-understanding multi-agent system showing better optimization budget efficiency and better performance scaling as the number of agents grows.

Main Findings

• Consistent gains on document understanding. With PE2 as the optimizer, MDocAgent improved on every one of the five document benchmarks, with an average gain of 4.3 points (65.2 to 69.5), including MMLongBench-Doc rising from 54.1 to 61.0 (+6.9), PaperTab from 46.3 to 53.7, PaperText from 72.1 to 76.4, LongDocURL from 74.9 to 75.4, and FetaTab from 78.8 to 80.8. DocAgent improved by an average of 2.7 points (70.0 to 72.7), with its largest gain on MMLongBench-Doc, 59.5 to 67.4 (+7.9).

• CEPO also lifts a multi-agent-specific optimizer. On top of MASPO for math and code, CEPO beat both the baseline and MASPO alone on all five benchmarks, reaching an average of 73.89 versus 72.85 for MASPO and 68.82 for the baseline (a 5.07-point average gain over baseline). Per-benchmark CEPO scores were AQuA 87.01, MATH-500 79.60, AGIEval-MATH 62.89, HumanEval-ET 74.39, and MBPP 65.57.

• Gains on long video understanding. On LongTVQA+, LongVideoAgent scored 60.3 unoptimized, 63.4 with PE2, and 64.9 with CEPO, a 1.5-point gain over vanilla PE2 and 4.6 points over baseline.

• Both prompt layers matter. In the component ablation on MDocAgent with MMLongBench-Doc, adding an LLM-generated coordinator without optimization raised accuracy from 54.1 to 55.6; optimizing only the executor with vanilla PE2 reached 57.9; optimizing only the coordinator reached 58.6; optimizing both with role-aligned guidance reached 61.0.

• Guidance must match the role. Swapping the meta-instructions degraded results: coordination guidance on the coordinator plus execution guidance on the executor scored 61.0 (best), coordination guidance on both scored 58.1, execution guidance on both scored 59.1, and reversing the assignment (execution guidance on the coordinator, coordination guidance on the executor) scored worst at 57.7.

• Better budget efficiency. PE2 + CEPO with one iteration per phase reached 60.1 accuracy using 2 optimizer API calls and 3,120 multi-agent API calls, exceeding vanilla PE2 with three executor iterations (57.9, 10 optimizer calls, 5,714 multi-agent calls). At comparable cost, CEPO with three iterations per phase reached 61.0 (20 optimizer calls, 11,458 calls) while vanilla PE2 with six executor iterations reached 57.4 (22 optimizer calls, 11,945 calls). Pushing CEPO to four iterations per phase dropped to 58.6.

• Performance scales with the number of agents. Across MDocAgent configurations of one to five agents, the unoptimized system gained little as agents were added and vanilla PE2 was non-monotonic, falling below the unoptimized result at three agents (driven by a drop on GENERAL + TEXT + SUM from 52.2 to 47.5). PE2 + CEPO increased monotonically to 61.0 at five agents, and its margin over the unoptimized system grew from 3.9 points at two agents to 6.9 at five. With only two agents, CEPO already exceeded the best unoptimized five-agent result.

• Works across optimizers. On MDocAgent across all five document benchmarks, CEPO improved PE2, ProTeGi (55.5 to 57.9 on MMLongBench-Doc), and OPRO (57.3 to 60.4 on MMLongBench-Doc), with gains of 0.6 to 6.5 points over the corresponding vanilla optimizer across optimizer-benchmark combinations.

• Sequential coordinator-first scheduling is best. With a fixed total budget of six iterations, sequential coordinator-first scored 61.0, sequential executor-first 60.7, alternating coordinator-first 60.1, and alternating executor-first 58.2.

• Which executor is optimized matters. Optimizing the GENERAL agent's executor gave 61.0 (versus 57.9 for PE2), TEXT gave 59.1 (versus 55.3), CRITIC 58.0 (versus 57.9), IMAGE 58.0 (versus 54.2), and SUM 55.8.

• Little inference overhead in the document setting. Throughput was 15.92 sec/sample for the baseline, 13.35 for PE2, and 15.61 for CEPO, with output tokens of 847.7, 595.8, and 824.7 respectively. A separate control extending vanilla PE2's prompts to a comparable length did not recover CEPO's performance.

Methodology in Plain English

The researchers start from the observation that a multi-agent prompt is doing two jobs at once: telling the team how to work together and telling one agent how to do its specific task. They split those jobs into two separately editable pieces.

The first piece is a single coordinator prompt, shared by every agent, that carries team-level rules such as how findings are shared, how disagreements are flagged, and what reasoning norms apply. The second piece is a set of executor prompts, one per agent, describing how that agent performs its assignment. Whenever an agent is invoked, its effective prompt is simply the coordinator prompt prepended to its own executor prompt. Because the coordinator is one shared object, a single edit to it propagates to the whole team at once, while executor edits stay local.

The second idea is to tell the optimizer what kind of improvement to look for. When updating the coordinator, a coordinator meta-instruction pushes the optimizer toward team-level signals: cross-agent communication, handling conflicting evidence, shared reasoning norms, and compact rules rather than formatting tweaks. When updating an executor, an executor meta-instruction pushes toward agent-level signals: task scoping, evidence use, concrete procedures, and procedural details that prevent recurring failures. Both layers are still judged by the same overall system score, so the meta-instructions change the guidance, not the objective.

The optimization runs in two phases. First, the coordinator is updated for a set number of iterations while all executor prompts are held at their initial values, and the best coordinator is chosen by validation score. Second, with that coordinator frozen, a chosen subset of executor prompts is updated one at a time, again selected by validation score. Setting the subset to empty gives coordinator-only optimization. Crucially, the framework never implements its own search: each update is a call to a user-supplied optimizer such as PE2, OPRO, ProTeGi, or MASPO, so CEPO is a wrapper around existing tools rather than a replacement.

Evaluation covers 11 benchmarks in four domains. Because the document benchmarks lack training sets, the authors split their test sets at the document level into 10/10/80 train/validation/test partitions with a fixed seed, keeping all questions for a document in the same split to avoid context leakage, and they verify that baseline scores on their split are close to the original test-set scores. Answers are graded by GPT-5-mini as an LLM judge. The agent systems studied are MDocAgent (Qwen-3.5 4B, a five-agent text/image pipeline) and DocAgent (GPT-5-mini, a three-agent actor-reviewer-reflector), MASPO's Qwen3-8B-based system for math and code, and LongVideoAgent (Qwen2.5-7B master with GPT-5-mini grounding and vision agents) for video. GPT-5.4 serves as the optimizer LLM for the document and video experiments. Executor prompts are initialized from each baseline system's published prompts, and the initial coordinator is generated by GPT-5-mini from a description of the system and its agent roles.

Why This Matters

Prompt optimization research has mostly treated multi-agent systems as a collection of independent prompts to be tuned jointly. This paper argues that the structure of what gets optimized matters as much as the optimizer, and shows that a simple two-layer split plus role-specific guidance improves four different optimizers, including one (MASPO) already designed specifically for multi-agent settings. That makes the contribution composable with future optimizers rather than competing with them.

The agent-count result is the more consequential one for practice: in this system, adding agents to an unoptimized pipeline bought almost nothing, and naive prompt optimization was unstable enough to fall below baseline at three agents. Explicit coordination optimization turned additional agents into monotonic gains. This speaks directly to the common practitioner experience that multi-agent pipelines are hard to scale past a few roles.

Real-world applications suggested by the evaluated domains:

• Document understanding and enterprise document QA, where teams of text, image, and summarization agents process long multimodal PDFs and reports.
• Code generation pipelines, where planning, programming, testing, and debugging roles must hand off work cleanly.
• Mathematical and analytical reasoning, where debate and iterative reflection among agents refine candidate answers.
• Long-video understanding, where grounding, temporal localization, and verification agents must combine multimodal evidence.

Industry relevance: CEPO reduces the manual prompt-engineering effort of keeping many agent prompts mutually consistent, since shared coordination rules live in one place. The budget-efficiency results matter commercially because the coordinator's gains were obtained with fewer optimizer and multi-agent API calls than a longer vanilla optimization run, and the paper reports that CEPO added little inference overhead relative to the baseline in the document setting. The authors also note a dual-use risk: easier prompt optimization may encourage deployment without adequate validation, and they recommend task-specific validation and human oversight.

Future Directions

• Scheduling and budget allocation. The paper reports that a sequential coordinator-first schedule beat alternating schedules at a fixed six-iteration budget, and that four iterations per phase underperformed three (58.6 versus 61.0). How to allocate a fixed budget across coordinator and executor phases, and when to stop, remains an open design question.

• Choosing which executors to optimize. The target-set ablation shows large variation by agent (GENERAL 61.0, SUM 55.8, and several agents far below that). Automatically selecting which agents deserve executor updates, rather than the user-specified subset CEPO currently takes, is a natural extension.

• Generalization beyond the evaluated systems. The evidence comes from MDocAgent, DocAgent, MASPO's Qwen3-8B system, and LongVideoAgent. Whether the decomposition holds for systems with different topologies, such as dynamically routed or debate-heavy pipelines, is not established here.

• Risk and validation. The ethics statement flags that optimized prompts can still produce unintended behavior in high-stakes or out-of-distribution settings. Methods for validating optimized multi-agent prompts before deployment, and for detecting coordination rules that overfit the validation split, are not addressed in the paper.

Target Audience

Researchers and practitioners working on LLM agent systems and automatic prompt optimization, especially those maintaining multi-agent pipelines where prompt changes must stay consistent across many roles. It is also useful for engineers deciding whether to invest in a multi-agent architecture at all, since the agent-count analysis quantifies when extra agents start paying off. Graduate students entering the prompt-optimization literature will find the related-work framing and the optimizer-agnostic design instructive, and the paper requires only moderate background: comfort with prompt optimization terminology and agent pipelines is enough to follow the method.

Authors’ abstract

Multi-agent systems (MAS) require both effective coordination and specialized task execution. Existing prompt optimization approaches typically couple these functions within agent-specific prompts. We introduce Coordinator-Executor Prompt Optimization (CEPO), a general framework for improving prompt optimization in MAS. CEPO separates system-level coordination from agent-level execution through a shared coordinator prompt and agent-specific executor prompts, and guides their updates according to their respective roles. This structure allows shared coordination rules to be refined consistently across agents while keeping execution instructions separately adjustable. CEPO provides the prompt structure and optimization guidance, while existing prompt optimizers perform the underlying search. Extensive experiments spanning 11 benchmarks across four domains demonstrate that applying CEPO to four existing prompt optimizers consistently improves their performance. Ablations and qualitative analyses support the proposed design, while controlled studies on a document-understanding MAS further demonstrate improved optimization budget efficiency and performance scaling with the number of agents.