Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting

AI summary

Overview

Research area: Security evaluation of LLM-based multi-agent systems (MAS) under adversarial prompting, bridging AI safety, agentic architecture design, and benchmark methodology.

Technical level: Intermediate. Readers need familiarity with LLM agents, orchestration/planning abstractions, and standard safety metrics (ASR, refusal rates), but the paper's core arguments are architectural and accessible.

Scope (1 sentence): The paper introduces a unified evaluation framework (SafeAgents) and a diagnostic metric (Dharma) to localize where inside a multi-agent pipeline adversarial prompts succeed or get rejected, then applies both across five MAS architectures and four safety benchmarks.

What This Paper Is About

Prior security work on LLM agents mostly studies single-agent settings, leaving unclear whether safety guarantees transfer when a task is decomposed across multiple specialized agents. The authors argue that a harmful request might be refused by a single agent but succeed when split among a planner and sub-agents, because the responsibility for recognizing harm is fragmented. Their goal is to systematically identify these "weak links" — the specific planning, delegation, or execution components that allow unsafe tasks to proceed — and to show how design choices rather than model alignment alone determine MAS safety.

Key Contributions

1. SafeAgents, a unified framework for architectural safety evaluation. A modular framework that integrates popular agentic frameworks (Magentic, LangGraph, OpenAI Agents), supports centralized and decentralized architectures, connects to complex backends (browsers, code execution environments, simulated file systems), and is extensible to new MAS and datasets. It abstracts agents, tools, and tasks into framework-agnostic Agent, Tool, Team, and Dataset classes, with Team.create() accepting framework selection and architecture type as parameters.

2. Dharma, a design-aware diagnostic metric. A hierarchical classification of rejection modes — Planner-Stop, Planner-Ignored, Planner-Failed, Sub-Agent-Stop, Sub-Agent-Ignored, Unmitigated Execution, and Error — that attributes refusals to specific components of a pipeline. It complements existing aggregate measures (Attack Success Rate, Refusal Rate) and the ARIA risk levels rather than replacing them.

3. An extensive empirical study. Five MAS architectures (Magentic-One, LangGraph, OpenAI Agents in centralized and decentralized variants) evaluated on four benchmarks (AgentHarm, ASB, SafeArena, RedCode) spanning code, web, and tool-use domains, with both a frontier model (gpt-4o) and a small language model (Qwen3-30B-A3B-Instruct-2507).

4. Evidence that design choices drive vulnerability, plus prompt-based mitigations. The study shows planning strategy, sub-agent autonomy, context organization, and fallback behavior shape safety outcomes, and that simple prompt-based mitigations already provide significant security benefits.

Main Findings

• Centralized MAS are not inherently safer than single-agent setups. Centralized systems sometimes show higher attack success than the single-agent baseline: ARIA-4 for centralized Magentic rises from 62.5% (single agent) to 83.70% in RedCode and from 81.5% to 92.52% in ASB. A single orchestrator can propagate undetected harmful intent more efficiently than a single agent or a decentralized system.

• Decentralized MAS perform better on context-heavy tasks. Decentralized Swarm achieves 15.85% ARIA-4 in SafeArena and outperforms centralized Magentic in RedCode (73% vs. 83.70%). Decentralized LangGraph reaches 17.6% ARIA-4 in SafeArena, lower than the single-agent and other centralized systems. The authors note this is domain dependent and does not generalize to all task types.

• Implementation choices within the same architectural family produce large security variation. Among centralized systems, Magentic reaches 83.70% ARIA-4 in RedCode while LangGraph and OpenAI Agents produce different results, so the architectural category alone does not predict security.

• Missing fallbacks when planning fails enable harmful execution. In Magentic, when the planner fails to generate a plan, the system lacks explicit instructions to halt and instead invokes downstream sub-agents. This appears in AgentHarm, where Magentic shows a 51.97% Sub-agent Stop classification — safety depends on downstream agents rather than respecting the planner's refusal.

• Atomic instruction delegation fragments context and hides harmful objectives. Magentic's WebSurfer receives isolated commands such as "open website" or "click button" without the higher-level task context, producing 66.37% Unmitigated-Execution classifications in SafeArena and elevated ARIA-4 risk.

• Stratified planning enables blind execution of pre-generated harmful plans. Once a plan exists, Magentic's orchestrator delegates based solely on it without reassessing harmfulness, contributing to the same 66.37% Unmitigated-Execution figure in SafeArena.

• Sub-agent refusal override is a real but less frequent failure. In RedCode, orchestrators sometimes continue execution through alternative delegation paths despite clear refusals from Coder or Computer Terminal agents, producing nonzero Sub-agent Ignore scores.

• Successful defenses exist and come from model alignment. AgentHarm shows 78.74% Planner-Stop for LangGraph and 87.40% for OpenAI Agents, because the tasks are explicitly harmful. Sub-agent refusals act as an effective secondary defense in RedCode, where coder sub-agents refuse to generate malicious code despite orchestrator approval.

• Unmitigated execution reveals a fundamental gap. A significant share of attack successes occur with no agent in the system recognizing or refusing harm, indicating that existing LLM alignment does not reliably transfer to multi-agent contexts.

• Small language models show an inverted failure pattern. With Qwen3-30B-A3B-Instruct-2507, planner-level defenses systematically fail: Planner-Stop drops to 0.00% for Magentic on AgentHarm (versus the much higher LangGraph and OpenAI GPT figures) and to 0.00% across all three frameworks on RedCode. Sub-agent defenses compensate on AgentHarm — Magentic reaches 81.89% Sub-agent Stop — but on RedCode both layers fail, yielding 99.26% (Magentic), 89.96% (LangGraph), and 93.65% (OpenAI Agents) Unmitigated-Execution.

Methodology in Plain English

The authors start by abstracting centralized and decentralized MAS into comparable design primitives: sub-agent autonomy (atomic instructions vs. higher-level subtasks), stratified vs. combined planning, and context organization (how plans, task descriptions, and agent roles are ordered in the prompt). This lets different frameworks be compared on equal footing even though their codebases differ substantially.

They then build SafeAgents, a framework-agnostic implementation layer where an Agent is configured through AgentConfig (tools, system messages, handoff targets), tools are defined via a decorator pattern that hides the underlying environment, and a Team class exposes a single create() factory that takes framework and architecture type as inputs, with each integrated framework subclassing Team and implementing run(). A Dataset abstraction defines task collections plus evaluation workflows, with built-in ARIA and Dharma assessment, attack detection, and checkpointing for long experiments.

Dharma works as a decision tree over execution trajectories. It first checks planner behavior (Planner-Stop, Planner-Ignored, Planner-Failed), then, if a valid plan was produced without refusal, sub-agent behavior (Sub-Agent-Stop, Sub-Agent-Ignored), and finally falls through to Unmitigated Execution or Error. Because manual labeling across many benchmarks is impractical, trajectories are classified by an LLM-as-judge using gpt-4.1 (version 2025-04-14, temperature 0) with framework-specific prompts. The MAS themselves run on gpt-4o (version 2024-11-20, temperature 0) for Tables 1 and 2, and on Qwen3 for Table 3, all at temperature 0. Reported metrics combine the aggregate ARIA levels with Dharma classes for a multi-resolution view.

Why This Matters

Impact on research. The paper shifts MAS security evaluation from outcome-only measurement to component-level attribution. By showing that the same final ARIA-4 outcome can arise from different root causes in Magentic versus LangGraph, it demonstrates that ASR, Refusal Rate, and ARIA alone cannot tell developers what to fix. It also provides a shared framework and metric so that future MAS safety results become comparable rather than custom per study.

Real-world applications:

• Healthcare and finance deployments, the domains the paper names as increasingly using MAS, where a missed refusal can cascade into significant real-world consequences.
• Code generation and execution agents, where RedCode-style attacks bypass safety at the planning stage for smaller models and produce near-total unmitigated execution.
• Web and browser agents, such as those operating on platforms resembling Reddit, GitLab, and e-commerce sites, where SafeArena-style context-heavy tasks favor decentralized designs.
• Tool-use agents, where ASB's prompt-injection threat model exposes weaknesses in communication and delegation pathways.

Industry relevance. Teams building on Magentic, LangGraph, or OpenAI Agents can use SafeAgents' declarative configuration to test design variations before deployment, and the finding that prompt-based mitigations help suggests a cheap first line of defense. The results also imply that security reviews of MAS should examine delegation granularity, fallback logic, and context sharing, not just the underlying model's alignment.

Future Directions

• Designing planner-level safety for small models. Because SLM-based MAS fail at planning while retaining execution-level refusal, the authors suggest either strengthening planning-level safety mechanisms or redesigning architectures to place safety checks at the execution layer.
• Establishing whether decentralized advantages generalize. The paper notes decentralized gains in SafeArena are domain dependent and do not generalize to all task types, leaving open which task properties determine the winner.
• Closing the unmitigated-execution gap. Since a large fraction of successful attacks involve no agent refusing at all, better mechanisms are needed when distributed context prevents any single agent from seeing the harmful objective.
• Standardizing cross-framework comparison. Extending SafeAgents to additional architectures and datasets, and validating the LLM-as-judge Dharma classifier more broadly, are natural next steps the framework's design invites.

Target Audience

This paper benefits MAS developers and architects choosing between centralized and decentralized patterns, AI safety and security researchers who need component-level rather than aggregate evaluation, benchmark designers working on agentic threat models, and engineering teams in regulated domains such as healthcare and finance that are deploying multi-agent pipelines and need to know where their defenses actually sit.

Authors’ abstract

LLM-based agents are increasingly deployed in multi-agent systems (MAS). As these systems move toward real-world applications, their security becomes paramount. Existing research largely evaluates single-agent security, leaving a critical gap in understanding the vulnerabilities introduced by multi-agent design. However, existing systems fall short due to lack of unified frameworks and metrics focusing on unique rejection modes in MAS. We present SafeAgents, a unified and extensible framework for fine-grained security assessment of MAS. SafeAgents systematically exposes how design choices such as plan construction strategies, inter-agent context sharing, and fallback behaviors affect susceptibility to adversarial prompting. We introduce Dharma, a diagnostic measure that helps identify weak links within multi-agent pipelines. Using SafeAgents, we conduct a comprehensive study across five widely adopted multi-agent architectures (centralized, decentralized, and hybrid variants) on four datasets spanning web tasks, tool use, and code generation. Our findings reveal that common design patterns carry significant vulnerabilities. For example, centralized systems that delegate only atomic instructions to sub-agents obscure harmful objectives, reducing robustness. Our results highlight the need for security-aware design in MAS. Link to code is https://github.com/microsoft/SafeAgents