# MCPSecBench: A Systematic Security Benchmark and Playground for Testing Model Context Protocols

arXiv: https://arxiv.org/abs/2508.13220 · HTML: https://mlsanity.com/paper/2508.13220/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 39 references that are in the map.
- First posted on arXiv 17 Aug 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 15 of 15)

- [Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models](https://mlsanity.com/paper/2312.14197/) (2023)
- [Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Servers](https://mlsanity.com/paper/2506.13538/) (2025)
- [MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits](https://mlsanity.com/paper/2504.03767/) (2025)
- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategies](https://mlsanity.com/paper/2504.08623/) (2025)
- [MCPWorld: A Unified Benchmarking Testbed for API, GUI, and Hybrid Computer Use Agents](https://mlsanity.com/paper/2506.07672/) (2025)
- [Firewalls to Secure Dynamic LLM Agentic Networks](https://mlsanity.com/paper/2502.01822/) (2025)
- [MCIP: Protecting MCP Safety via Model Contextual Integrity Protocol](https://mlsanity.com/paper/2505.14590/) (2025)
- [MCP Guardian: A Security-First Layer for Safeguarding MCP-Based AI System](https://mlsanity.com/paper/2504.12757/) (2025)
- [We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems](https://mlsanity.com/paper/2506.13666/) (2025)
- [ETDI: Mitigating Tool Squatting and Rug Pull Attacks in Model Context Protocol (MCP) by using OAuth-Enhanced Tool Definitions and Policy-Based Access Control](https://mlsanity.com/paper/2506.01333/) (2025)
- [MCP-Guard: A Multi-Stage Defense-in-Depth Framework for Securing Model Context Protocol in Agentic AI](https://mlsanity.com/paper/2508.10991/) (2025)
- [We Urgently Need Privilege Management in MCP: A Measurement of API Usage in MCP Ecosystems](https://mlsanity.com/paper/2507.06250/) (2025)
- [A Survey on Agent Workflow -- Status and Future](https://mlsanity.com/paper/2508.01186/) (2025)
- [A Large-Scale Evolvable Dataset for Model Context Protocol Ecosystem and Security Analysis](https://mlsanity.com/paper/2506.23474/) (2025)

## Built on by (most cited first, 30 of 51)

- [AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocations](https://mlsanity.com/paper/2603.10749/) (2026)
- [MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers](https://mlsanity.com/paper/2512.15163/) (2025)
- [Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems](https://mlsanity.com/paper/2601.17548/) (2026)
- [Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges](https://mlsanity.com/paper/2510.23883/) (2025)
- [Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents](https://mlsanity.com/paper/2601.17549/) (2026)
- [VIGIL: Defending LLM Agents Against Tool Stream Injection via Verify-Before-Commit](https://mlsanity.com/paper/2601.05755/) (2026)
- [Are Your Agents Upward Deceivers?](https://mlsanity.com/paper/2512.04864/) (2025)
- [TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories](https://mlsanity.com/paper/2604.07223/) (2026)
- [AIRGuard: Guarding Agent Actions with Runtime Authority Control](https://mlsanity.com/paper/2605.28914/) (2026)
- [Model Context Protocol Threat Modeling and Analyzing Vulnerabilities to Prompt Injection with Tool Poisoning](https://mlsanity.com/paper/2603.22489/) (2026)
- [AgentAtlas: Beyond Outcome Leaderboards for LLM Agents](https://mlsanity.com/paper/2605.20530/) (2026)
- [Lingering Authority: Revocable Resource-and-Effect Capabilities for Coding Agents](https://mlsanity.com/paper/2606.22504/) (2026)
- [Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation](https://mlsanity.com/paper/2606.10749/) (2026)
- [Agentic Services Computing](https://mlsanity.com/paper/2509.24380/) (2025)
- [Trusted AI Agents in the Cloud](https://mlsanity.com/paper/2512.05951/) (2025)
- [SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction](https://mlsanity.com/paper/2606.02540/) (2026)
- [Formalizing the Safety, Security, and Functional Properties of Agentic AI Systems](https://mlsanity.com/paper/2510.14133/) (2025)
- [SMCP: Secure Model Context Protocol](https://mlsanity.com/paper/2602.01129/) (2026)
- [Don't believe everything you read: Understanding and Measuring MCP Behavior under Misleading Tool Descriptions](https://mlsanity.com/paper/2602.03580/) (2026)
- [AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP](https://mlsanity.com/paper/2607.11098/) (2026)
- [MCPZoo: A Large-Scale Dataset of Runnable Model Context Protocol Servers for AI Agent](https://mlsanity.com/paper/2512.15144/) (2025)
- [MCPShield: A Security Cognition Layer for Adaptive Trust Calibration in Model Context Protocol Agents](https://mlsanity.com/paper/2602.14281/) (2026)
- [A Formal Security Framework for MCP-Based AI Agents: Threat Taxonomy, Verification Models, and Defense Mechanisms](https://mlsanity.com/paper/2604.05969/) (2026)
- [HarnessSafe: Evaluating Safety Across Persistent Carriers in Agent Harnesses](https://mlsanity.com/paper/2608.06984/) (2026)
- [MCP-DPT: A Defense-Placement Taxonomy and Coverage Analysis for Model Context Protocol Security](https://mlsanity.com/paper/2604.07551/) (2026)
- [Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implications](https://mlsanity.com/paper/2606.04769/) (2026)
- [Can LLM Infer Risk Information From MCP Server System Logs?](https://mlsanity.com/paper/2511.05867/) (2025)
- [Give Them an Inch and They Will Take a Mile:Understanding and Measuring Caller Identity Confusion in MCP-Based AI Systems](https://mlsanity.com/paper/2603.07473/) (2026)
- [ShieldNet: Network-Level Guardrails against Emerging Supply-Chain Injections in Agentic Systems](https://mlsanity.com/paper/2604.04426/) (2026)
- [MCPHunt: An Evaluation Framework for Cross-Boundary Data Propagation in Multi-Server MCP Agents](https://mlsanity.com/paper/2604.27819/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly integrated into real-world applications via the Model Context Protocol (MCP), a universal open standard for connecting AI agents with data sources and external tools. While MCP enhances the capabilities of LLM-based agents, it also introduces new security risks and significantly expands their attack surface. In this paper, we present the first formalization of a secure MCP and its required specifications. Based on this foundation, we establish a comprehensive MCP security taxonomy that extends existing models by incorporating protocol-level and host-side threats, identifying 17 distinct attack types across four primary attack surfaces. Building on these specifications, we introduce MCPSecBench, a systematic security benchmark and playground that integrates prompt datasets, MCP servers, MCP clients, attack scripts, a GUI test harness, and protection mechanisms to evaluate these threats across three major MCP platforms. MCPSecBench is designed to be modular and extensible, allowing researchers to incorporate custom implementations of clients, servers, and transport protocols for rigorous assessment. Our evaluation across three major MCP platforms reveals that all attack surfaces yield successful compromises. Core vulnerabilities universally affect Claude, OpenAI, and Cursor, while server-side and specific client-side attacks exhibit considerable variability across different hosts and models. Furthermore, current protection mechanisms proved largely ineffective, achieving an average success rate of less than 30%. Overall, MCPSecBench standardizes the evaluation of MCP security and enables rigorous testing across all protocol layers.