EvoTeam: Multi-Agent Collaboration via Evolving Teams

AI summary

Overview

Research area: LLM-based multi-agent systems (MAS), specifically experience-driven self-evolution of how agents are organized and how they coordinate.

Technical level: Advanced. The paper assumes familiarity with LLM agent scaffolds, tool-use environments, benchmark suites such as GAIA, and prior evolution methods (Meta-Team, Skill-MAS, ReCreate, AgentSquare).

Scope (one sentence): EvoTeam is a framework that maintains a library of reusable agent teams, gives each team its own evolving interaction rules, and lets individual agent skills carry over across teams, so that collaboration experience accumulated on one kind of task can be selectively reused on heterogeneous tasks.

What This Paper Is About

Most work on improving multi-agent systems makes individual agents better by refining their prompts, skills, and reasoning strategies from past execution feedback. The authors argue that better individual agents do not automatically produce better teams, because collaboration depends on how responsibilities are divided, how information is handed off, and when the team stops. EvoTeam addresses this by evolving collaboration itself: it selects or creates a task-appropriate team, coordinates work through rules scoped to that specific team, and then uses execution traces to refine the team's roster, its rules, and each agent's skills.

Key Contributions

1. Problem framing. The authors identify two linked obstacles to collaboration-centric evolution: constrained team reorganization (existing methods adjust membership and responsibilities inside one shared team structure) and inappropriate reuse of collaboration experience (interaction rules learned under one set of responsibilities are applied in incompatible team contexts).

2. A team-indexed evolution framework. EvoTeam maintains a global agent pool and a library of reusable teams, where each team is a description of the collaboration requirements it supports plus a candidate agent roster. Compatible tasks reuse and refine a team's rules; agent skills stay portable across teams.

3. Three coordinated components. Adaptive Agent Team Composition (a Composer, a Team Selector, and a Chairman that recruits an execution team), Team-Scoped Collaboration (intra-team planning with subtask assignment plus team-specific rules injected as soft guidance), and Multi-Level Experience Evolution (a Team Reflector, a Rule Reflector, and an Agent Reflector operating on different parts of the execution trace).

4. Broad empirical validation. Experiments cover four benchmarks and six task categories under both individual-benchmark and mixed-benchmark evolution, plus cross-backbone transfer between DeepSeek-V4.1-Flash and GPT-5.6-Luna, ablations, cost–performance comparisons, and case analyses.

Main Findings

• Prior experience can hurt. In the preliminary study, both cross-task settings (Inter-CT, starting from another benchmark with an unsuitable team template, and Intra-CT, starting from another task category in the same benchmark with a compatible template) underperformed target-only Meta-Team on all three reported metrics. The paper's Figure 2(b) traces this to a GAIA-oriented team failing to establish an implementation-and-testing process for a LoCoBench PII-anonymization task, and to a diagnosis-oriented rule that tells the file agent to report findings and finalize, which can terminate execution before coding and testing.

• Best average under individual-benchmark evolution. EvoTeam reaches an average score of 58.85, leading on four of six metrics (GAIA, LoCoBench Feat., LoCoBench Refact., BeyondSWE DepMig.) and outperforming the unevolved MAS baseline on every metric. The GAIA margin over Meta-Team is 78.00 versus 73.00 (five points).

• Stronger and more stable under mixed-benchmark evolution. EvoTeam ranks first on all six metrics with an average of 59.58, versus 55.55 for Meta-Team, a gap of 4.03 points. EvoTeam's average improves relative to its individual-benchmark result (58.85 to 59.58), whereas Meta-Team's declines (58.30 to 55.55).

• Evolved collaboration transfers across LLM backbones. Cross-backbone, EvoTeam beats the unevolved MAS baseline in all six configurations. On GAIA, gains are 5.00 points for GPT-5.6-Luna evolved by DeepSeek (72.00 to 77.00) and 18.00 points for DeepSeek-V4.1-Flash evolved by GPT (58.00 to 76.00). Same-backbone gains are +22.00 (DeepSeek, 58.00 to 80.00) and +6.00 (GPT, 72.00 to 78.00).

• Cross-backbone performance stays close to same-backbone. Across all six paired comparisons, cross-backbone results remain within 4.00 points of the corresponding same-backbone result; on GAIA, cross-backbone reaches 77.00 with GPT-5.6-Luna versus 78.00 same-backbone.

• Interaction rules matter, especially on GAIA and LOCA. Removing learned rules while keeping the task-conditioned team costs 10.00 points on GAIA and 8.75 points on LOCA. Gains are smaller on the coding tasks, which the authors read as task-dependent value of coordination guidance.

• Task-conditioned teams help even with no rules. With rules absent from both variants, retaining the team structure still improves every task, by 6.00 points on GAIA and 2.50 on LOCA.

• Favorable cost–performance position. On GAIA and BeyondSWE DepMig under mixed-benchmark evolution, EvoTeam achieves the highest score while costing less per task than Meta-Team and ReCreate; Skill-MAS is cheaper but scores lower. The paper does not report the numeric per-task costs in the visible text.

Methodology in Plain English

The system keeps two stores: a pool of agents, each carrying its own evolving set of skills, and a library of reusable teams. A team is not a fixed group of agents but a reusable organization: a written description of what kind of collaboration it supports, plus a roster of candidate agents who could staff it.

For each incoming task, a Team Selector checks whether an existing team's description covers the task's collaboration needs and whether that team's roster contains the right functional roles. If nothing fits, a Composer builds a provisional team from the agent pool; that team is only added to the library permanently after execution, if it proves both capable and reusable. A Chairman then recruits a task-specific execution team from the chosen roster — different tasks reusing the same team can recruit different subsets — and splits the task into subtasks with an assignment plan. Crucially, the plan sets responsibilities but does not dictate who talks to whom.

During execution, agents use their accumulated skills and consult the rules attached to the selected team, injected into their context as soft guidance rather than hard scripts. Agents choose which rule applies to their current coordination need. The Chairman monitors progress and can reassign subtasks if the division of labor stalls. Everything is logged as a composite trace: per-agent traces (reasoning, tool outputs, artifacts) plus a handoff trace (interaction events, transferred payloads, rules used).

After the task, three reflectors act at different levels: the Team Reflector checks whether the roster supported the required division of work and revises the roster and description if not (also deciding whether a provisional team is admitted); the Rule Reflector reads the handoff trace and adds, edits, or deletes interaction rules for that team only; the Agent Reflector reads each agent's own trace and adds or revises that agent's skills. Rules stay bound to their team, while skills stay bound to their agent and thus travel across teams. After the final evolution query, all three stores are frozen and evaluated on disjoint held-out splits.

Why This Matters

Research impact. The paper reframes MAS self-improvement from "make each agent smarter" to "make the unit of learning match the unit of collaboration." Its preliminary study is a useful negative result: carrying prior team templates and rules across heterogeneous tasks can actively degrade performance, which challenges the common assumption that accumulated experience is monotonically helpful. Scoping interaction rules to teams rather than to a global system gives a concrete mechanism for avoiding negative transfer.

Real-world applications.

• Software engineering pipelines: repository-level maintenance and cross-file refactoring, where the paper evaluates on LoCoBench Feat./Refact. and BeyondSWE DepMig./CrossR.
• Web research and information synthesis: multi-step browsing and retrieval assistants, evaluated on GAIA's 120-instance validation subset.
• Tool-driven productivity workflows: stateful application interaction via MCP, evaluated on LOCA-Bench, where a reusable team with evolving handoff rules suits repeated but varied office-style tasks.
• Enterprise agent platforms serving mixed request streams: the mixed-benchmark setting is the closest analogue to a production queue in which one evolving system must handle heterogeneous request types, and it is precisely where EvoTeam's average improves while Meta-Team's declines.

Industry relevance. The cost analysis positions EvoTeam ahead of Meta-Team and ReCreate on score per task cost for GAIA and BeyondSWE DepMig, and cross-backbone transfer (gains of 5.00 to 18.00 points on GAIA when evolution and deployment LLMs differ) suggests teams and rules evolved with one model remain useful after a model swap — a practical concern for teams that cannot re-run evolution every time a backend changes.

Future Directions

1. Multimodal and interactive environments. The limitations section states that collaborative evolution in multimodal interactive settings is not systematically examined, and asks how multimodal execution traces could guide refinement of team organization, rules, and skills.

2. Combining explicit experience with model adaptation. EvoTeam keeps LLM parameters fixed and stores experience as explicit text; the authors note that effective use depends on the base model correctly interpreting that stored guidance, and suggest pairing experience reuse with model adaptation to internalize frequently useful coordination practices.

3. Governance of accumulated experience. The ethics discussion flags that execution traces and reusable knowledge can retain sensitive information, motivating access controls, privacy protection, review of rules and skills before reuse, and human oversight for consequential actions.

4. Cost and scaling behavior beyond the two profiled tasks. Cost–performance is reported for GAIA and BeyondSWE DepMig under mixed-benchmark evolution, with the individual-benchmark cost comparison placed in an appendix; how the team library and rule sets scale as tasks accumulate is not characterized in the visible content.

Target Audience

Researchers and graduate students working on LLM agent architectures, agent self-improvement, and multi-agent orchestration; engineers building production agent platforms that must serve heterogeneous task streams and periodically change underlying models; and readers interested in negative transfer and experience-reuse design in systems that learn from their own execution traces. Beginners will grasp the motivation but will need background in agent scaffolds and benchmarks such as GAIA to follow the experimental sections.

Authors’ abstract

LLM-based multi-agent systems (MAS) rely on collaboration among specialized agents to solve complex tasks. Existing methods improve MAS by refining the prompts, skills, and execution strategies of individual agents based on task feedback. However, improving individual agents does not necessarily lead to better team collaboration. Recent studies have therefore extended evolution to the collaboration level, using collective execution experience to adjust team organization, and inter-agent interactions. Nevertheless, evolving collaboration across heterogeneous tasks remains challenging, as existing approaches often constrain team reorganization within a shared structure and reuse interaction rules across incompatible team contexts. In this paper, we propose EvoTeam, a collaboration-centric evolution framework that jointly evolves team organization, interaction rules, and individual agent skills through execution experience. It maintains a library of reusable teams, selecting or creating agent teams and guiding collaboration through team-specific interaction rules. Collaboration experience accumulates within compatible team contexts, while individual skills remain reusable across teams. Experiments across four benchmarks demonstrate its effectiveness. Our code is available at https://github.com/DEEP-JLU/EvoTeam.