MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers

AI summary

Overview

Research area: Safety and security evaluation of large language model (LLM) agents that operate external tools through the Model Context Protocol (MCP).

Technical level: Advanced. The paper assumes familiarity with agentic LLM frameworks (ReAct-style agents), tool-calling protocols, benchmark design, and inferential statistics (t-tests, Mann–Whitney U, ANOVA, Cohen's d).

Scope in one sentence: The paper introduces MCP-SafetyBench, a 245-case, execution-based benchmark built on real MCP servers that measures both whether an agent completes a task and whether an injected MCP attack succeeds, then uses it to evaluate 13 open- and closed-source LLMs across 20 attack types and five domains.

What This Paper Is About

As LLMs shift from text generators to agents that plan and call external tools, MCP has become a standard way to connect them to heterogeneous servers. That openness creates safety risks — malicious tool descriptions, poisoned context across servers, forged identities, and data exfiltration — that existing MCP safety benchmarks largely fail to capture because they test isolated, one-shot attacks or are not grounded in real servers. This paper builds a benchmark on real-world MCP servers that supports multi-turn, cross-server evaluation, and uses it to measure how vulnerable current models actually are.

Key Contributions

1. A unified taxonomy of 20 MCP attack types organized across three sides — MCP Server, MCP Host, and User — consolidating and extending attack categories from prior work (e.g., tool poisoning variants, function overlapping, intent injection, credential theft).
2. MCP-SafetyBench itself: a benchmark built on the MCP-Universe benchmark using real-world MCP servers, supporting multi-step, multi-server tasks across five domains (browser automation, financial analysis, location navigation, repository management, web search) with 245 test cases.
3. A dual-label evaluation design: every task is paired with exactly one attack and scored twice — a task evaluator inherited from MCP-Universe for goal achievement, and a newly introduced attack evaluator with dedicated detectors (e.g., check_parameter_modification_attack, check_command_injection_attack, check_replay_injection_attack).
4. A systematic evaluation of leading proprietary and open-source LLMs, revealing that all evaluated models remain vulnerable, that attack effectiveness varies sharply by type and domain, and that a prompt-level safety mitigation provides only limited benefit.

Main Findings

• No model is immune. Across the 13 evaluated models, overall Attack Success Rate (ASR) ranges from 29.80% for Qwen3-235B to 48.16% for o4-mini. Overall Task Success Rate (TSR) ranges from 8.98% (GPT-4o) to 21.22% (o4-mini).

• A safety-utility trade-off appears to exist. Task Success Rate and Defense Success Rate (DSR = 1 − ASR) correlate negatively, with a Pearson coefficient of r = −0.572 (p = 0.041). The paper's example: o4-mini has the highest TSR (21.22%) but a relatively low DSR (51.84%), while Qwen3-235B has a lower TSR (10.20%) but a higher DSR (70.20%). The authors attribute this to high-performing models being optimized for precise instruction-following, which makes them more likely to follow instructions indiscriminately.

• Vulnerability differs significantly by domain. Financial Analysis is the most vulnerable domain, with an average ASR of 46.59% (Gemini-2.5-Flash reaches 56.60%). Web Search is the least vulnerable, at 30.33%. A one-way ANOVA confirms significant variation (F = 6.68, p = 0.000163, η² = 0.308). Pairwise comparisons: Financial Analysis exceeds the other domains' mean by Δ = +8.82% (p = 0.000010, Cohen's d = 1.87), and Web Search falls below it by Δ = −11.50% (p = 0.002559, Cohen's d = −0.95).

• Host-side attacks are the most effective. They average an 81.94% success rate. Identity Injection succeeds 100% of the time across all 13 tested models, described as a universal vulnerability.

• Tool-poisoning attacks vary internally. Tool Redirection reaches 70.63% ASR, while other tool-poisoning attacks average only 19.05%. Models also show strong resistance to Remote Access Control (13.08% ASR).

• Defenses are "spiky," not uniform. 76.9% of models (10 of 13) are strongly resistant to some attack types (e.g., Network Request Poisoning, File System Poisoning) while significantly vulnerable to others (e.g., Identity Injection, Intent Injection).

• Reasoning models are not safer. No significant ASR difference between reasoning and non-reasoning models: two-sample t-test p = 0.7778, Mann–Whitney U p = 0.8835, effect size |d| = 0.1648. Per-domain ASR examples: Financial Analysis 46.7% vs. 46.4%, Web Search 29.7% vs. 31.3%, Browser Automation 34.2% vs. 40.0%.

• Open-source vs. proprietary shows no systematic pattern. Two-sample t-test p = 0.4008, Mann–Whitney U p = 0.4398, |d| = 0.5252. Closed-source models show higher ASR in Location Navigation (44.4% vs. 39.6%), Financial Analysis (47.8% vs. 43.9%), and Repository Management (44.0% vs. 36.2%), while open-source models show higher ASR in Browser Automation (39.2% vs. 35.2%) and comparable Web Search (30.7% vs. 30.2%).

• Safety prompts give limited help. A prepended Safety Prompt reduces weighted ASR from 39.88% to 38.65% (−1.22%), which is not statistically significant (p = 0.2908, Cohen's d = 0.31). It helps significantly on some attacks — Malicious Code Execution (−21.54%, p = 0.0016), Credential Theft (−21.37%, p = 0.0027), Remote Access Control (−10.77%, p = 0.0093) — but is ineffective or harmful for others (Preference Manipulation +7.34%, Function Overlapping +9.36%). Benefits accrue mostly to proprietary models (e.g., Gemini, GPT series); open-source models show negligible or negative effects.

Methodology in Plain English

The researchers started from an existing set of real-world agent tasks (the MCP-Universe benchmark) and turned each one into a security test case through three steps.

First, task selection: tasks from five domains were adapted as "clean baselines," each preserving a goal, a context, available tools, and a machine-checkable output schema so correctness could be graded automatically.

Second, attack instantiation: each baseline was paired with exactly one attack from the taxonomy, applied at the appropriate level. Server-side attacks modify tool manifests or implementations (e.g., parameter poisoning, function-return injection); host-side attacks modify the host pipeline (e.g., intent rewriting, replay, identity spoofing); user-side attacks embed prompt-injection fragments directly into the user query. Attack examples were drafted with compact templates, synthesized with Cursor, and kept only after human review for plausibility and feasibility.

Third, task formalization and packaging: each task became a tuple of goal, context, available tools, and the injected attack, packaged into a manifest carrying the attack category (Disruption or Stealth), the user query, the output schema, attack metadata, and evaluators.

Evaluation is fully automated and execution-based. Every case runs through a standardized MCP pipeline with a ReAct-style agent that interleaves reasoning and acting. The agent's full trace is logged, then both a task evaluator (format, static, and dynamic checks inherited from MCP-Universe) and an attack evaluator (detectors looking for the success signature of the specific attack) are run on that trace. The result is a dual label: Task Outcome (Pass/Fail) and Attack Outcome (Success/Failure), aggregated into Task Success Rate (TSR) and Attack Success Rate (ASR).

All models ran under one configuration: temperature 1.0, maximum output length 2048 tokens, per-call timeout 60 seconds, up to 20 ReAct iterations per task, and 3 repetitions per task.

Benchmark composition: 245 cases — Financial Analysis 53, Location Navigation 53, Repository Management 56, Web Search 53, and Browser Automation 30 (fewer because these interactive tasks were harder to construct). By strategy, the benchmark is nearly evenly split between Disruption attacks (46.53%, aimed at causing task failure) and Stealth attacks (53.47%, aimed at achieving a malicious goal without alerting the user). By source, 74.69% of attacks originate from the MCP Server, 13.06% from the User, and 12.24% from the MCP Host. The most frequent individual types include Tool Poisoning Command Injection (12.65%) and Function Overlapping (9.39%).

Models evaluated: proprietary — GPT-5, GPT-4.1, GPT-4o, o4-mini, Claude-4.0-Sonnet, Claude-3.7-Sonnet, Gemini-2.5-Pro, Gemini-2.5-Flash, Grok-4; open-source — GLM-4.5, Kimi-K2, Qwen3-235B, DeepSeek-V3.1.

Why This Matters

Impact on research. The paper argues that prior MCP safety benchmarks either focus narrowly on specific attack types, lack real-world integration, or do not capture multi-turn dynamics. Table 1 in the paper positions MCP-SafetyBench as the only benchmark among the compared set (SafeMCP, MCPTox, MCIP-bench, MCP-AttackBench, MCPSecBench) that combines real-world integration, multi-step tasks, and coverage of server-, host-, and user-side attacks. It also connects its taxonomy to the broader OTM threat framework, mapping user-side attacks to the Application Input Layer, server-side attacks to the Context Data Layer, and host-side vulnerabilities to internal logic attacks.

Real-world applications:

• Third-party tool ecosystems: the finding that 74.69% of attacks originate server-side speaks directly to deployments where agents must trust numerous third-party MCP servers.
• Financial agents: Financial Analysis was the most vulnerable domain (average ASR 46.59%), relevant to any deployment that queries market data or executes financial operations.
• Enterprise agent platforms: Identity Injection's 100% success rate and the 81.94% average for host-side attacks point to risks in intent parsing and state management in orchestration layers.
• Retrieval and data pipelines: user-side attacks such as Retrieval-Agent Deception and Credential Theft target the data sources and credentials agents routinely touch.

Industry relevance. Because the benchmark is built on real MCP servers rather than simulated ones and uses deterministic, execution-based scoring, its results are intended to translate into deployment-level diagnostics. The paper notes that the safety-utility trade-off and the ineffectiveness of prompt-only defenses mean capability gains do not automatically buy security.

Future Directions

1. Multi-layered defenses beyond prompt-level safeguards, including robust model unlearning techniques to eradicate malicious attack patterns rather than suppress them at inference time.
2. Dynamic tool vetting for real-time mitigation, checking tools as they are discovered rather than relying on static allowlists.
3. Formalizing safe MCP behavior through contextual least privilege, such as privilege narrowing and context checking, so high-privilege tools cannot be invoked for low-privilege tasks.
4. Automated, adaptive defenses and benchmark expansion to broader real-world scenarios, aimed at securing long-horizon LLM agents in multi-tool environments.

Target Audience

This paper is most valuable to AI safety and security researchers working on agentic LLM systems, MCP and tool-integration platform engineers, and benchmark designers who need realistic, execution-based evaluation rather than static prompt tests. It is also relevant to product and risk teams deploying agents against third-party tool servers, and to model developers who want to understand why stronger instruction-following can correlate with weaker attack resistance. Readers should be comfortable with agent architectures, evaluation metrics, and statistical significance testing.

Authors’ abstract

Large language models (LLMs) are evolving into agentic systems that reason, plan, and operate external tools. The Model Context Protocol (MCP) is a key enabler of this transition, offering a standardized interface for connecting LLMs with heterogeneous tools and services. Yet MCP's openness and multi-server workflows introduce new safety risks that existing benchmarks fail to capture, as they focus on isolated attacks or lack real-world coverage. We present MCP-SafetyBench, a comprehensive benchmark built on real MCP servers that supports realistic multi-turn evaluation across five domains: browser automation, financial analysis, location navigation, repository management, and web search. It incorporates a unified taxonomy of 20 MCP attack types spanning server, host, and user sides, and includes tasks requiring multi-step reasoning and cross-server coordination under uncertainty. Using MCP-SafetyBench, we systematically evaluate leading open- and closed-source LLMs, revealing that all models remain vulnerable to MCP attacks, with a notable safety-utility trade-off. Our results highlight the urgent need for stronger defenses and establish MCP-SafetyBench as a foundation for diagnosing and mitigating safety risks in real-world MCP deployments.