mlsanityData to 14 Oct 2026

MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents

arXiv 2510.15994 ↗First posted 14 Oct 2025cs.CR cs.AIcited by 42

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

The Model Context Protocol (MCP) standardizes how large language model (LLM) agents discover, describe, and call external tools. While MCP unlocks broad interoperability, it also enlarges the attack surface by making tools first-class, composable objects with natural-language metadata, and standardized I/O. We present MSB (MCP Security Benchmark), the first end-to-end evaluation suite that systematically measures how well LLM agents resist MCP-specific attacks throughout the full tool-use pipeline: task planning, tool invocation, and response handling. MSB contributes: (1) a taxonomy of 12 attacks including name-collision, preference manipulation, prompt injections embedded in tool descriptions, out-of-scope parameter requests, user-impersonating responses, false-error escalation, tool-transfer, retrieval injection, and mixed attacks; (2) an evaluation harness that executes attacks by running real tools (both benign and malicious) via MCP rather than simulation; and (3) a robustness metric that quantifies the trade-off between security and performance: Net Resilient Performance (NRP). We evaluate nine popular LLM agents across 10 domains and 405 tools, producing 2,000 attack instances. Results reveal the effectiveness of attacks against each stage of MCP. Models with stronger performance are more vulnerable to attacks due to their outstanding tool calling and instruction following capabilities. MSB provides a practical baseline for researchers and practitioners to study, compare, and harden MCP agents. Code: https://github.com/dongsenzhang/MSB

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 42 of its 57 references that are in the map. First posted on arXiv 14 Oct 2025.

Builds on

42 of 57 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers19 Aug
  2. 2025MCPXKIT: The Unified Toolkit for Analyzing Model Context Protocol Security18 Aug
  3. 2025Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigms8 Jul
  4. 2025RAS-Eval: A Comprehensive Benchmark for Security Evaluation of LLM Agents in Real-World Environments18 Jun
  5. 2025We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems16 Jun
  6. 2025MCP Safety Training: Learning to Refuse Falsely Benign MCP Exploits using Improved Preference Alignment29 May
  7. 2025MPMA: Preference Manipulation Attack Against Model Context Protocol16 May
  8. 2025MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits2 Apr
  9. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agents19 Dec
  10. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  11. 2024EnIGMA: Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities24 Sep
  12. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents19 Jun
2 more
  1. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  2. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 28

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  6. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  7. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  8. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  9. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 392
  10. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 356
  11. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 278
  12. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 265
16 more
  1. 2023Inverse Scaling: When Bigger Isn't Bettercited by 207
  2. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 160
  3. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 153
  4. 2024Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigationcited by 126
  5. 2025Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolutioncited by 116
  6. 2025Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Serverscited by 65
  7. 2024Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usagecited by 51
  8. 2024API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMscited by 39
  9. 2025MCIP: Protecting MCP Safety via Model Contextual Integrity Protocolcited by 34
  10. 2025MCP-Zero: Active Tool Discovery for Autonomous LLM Agentscited by 33
  11. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 32
  12. 2025AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxescited by 30
  13. 2024From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Modelscited by 27
  14. 2024SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Modelscited by 23
  15. 2025We Urgently Need Privilege Management in MCP: A Measurement of API Usage in MCP Ecosystemscited by 15
  16. 2024Wait, that's not an option: LLMs Robustness with Incorrect Multiple-Choice Optionscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 42.

42 papers.

Sort:

By area:

  1. 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injectioncited by 27
  2. 2026Security Threat Modeling for Emerging AI-Agent Protocols: A Comparative Analysis of MCP, A2A, Agora, and ANPcited by 15
  3. 2026AIRGuard: Guarding Agent Actions with Runtime Authority Controlcited by 13
  4. 2026AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skillscited by 12
  5. 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Frameworkcited by 11
  6. 2026Model Context Protocol Threat Modeling and Analyzing Vulnerabilities to Prompt Injection with Tool Poisoningcited by 10
  7. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 7
  8. 2026MCP-38: A Comprehensive Threat Taxonomy for Model Context Protocol Systems (v1.0)cited by 6
  9. 2026MCPShield: A Security Cognition Layer for Adaptive Trust Calibration in Model Context Protocol Agentscited by 3
  10. 2026Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligencecited by 3
  11. 2026Invisible Threats from Model Context Protocol: Generating Stealthy Injection Payload via Tree-based Adaptive Searchcited by 2
  12. 2026MCP-DPT: A Defense-Placement Taxonomy and Coverage Analysis for Model Context Protocol Securitycited by 2
30 more
  1. 2026Red-Teaming Agent Execution Contexts: Open-World Security Evaluation on OpenClawcited by 2
  2. 2026ShieldNet: Network-Level Guardrails against Emerging Supply-Chain Injections in Agentic Systemscited by 1
  3. 2026MCPHunt: An Evaluation Framework for Cross-Boundary Data Propagation in Multi-Server MCP Agentscited by 1
  4. 2026When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agentscited by 1
  5. 2026GIF: Locally Sound Geometric Information Flow Control for LLMscited by 1
  6. 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directionscited by 1
  7. 2026ToolGuardian: Declarative Security for AI Agent-Tool Interactionscited by 1
  8. 2026Blockchain Empowered Trustworthy Agent Networks: Foundations, Taxonomy, and Future Directionscited by 1
  9. 2026MCPThreatHive: Automated Threat Intelligence for Model Context Protocol Ecosystemscited by 0
  10. 2026Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliabilitycited by 0
  11. 2026Comment and Control: Hijacking Agentic Workflows via Context-Grounded Evolutioncited by 0
  12. 2026Securing Multi-Tool AI Agent Chains With Dynamic, Real-Time Compositional Policiescited by 0
  13. 2026DynamicMCPBench: A Trace-Grounded, Effect-Scored Benchmark for LLM Agents over Live MCP Serverscited by 0
  14. 2026Hardware Keystores for AI Agent Signing Workflows: A Zero-Trust MCP Enforcement Architecturecited by 0
  15. 2026Labels Are Not Endpoints: Treatment Leakage and Construct Validity in MCP Agent Security Evaluationcited by 0
  16. 2026When Agents Act on Web3: An Attack-Surface Survey of MCP, Skills, and Tool Callingcited by 0
  17. 2026TrustShiftProbe: Characterizing, Benchmarking, and Defending Staged Trust Attacks on MCP Serverscited by 0
  18. 2026Public-Sharing Labels and Verbatim Field Egress in an MCP-to-A2A Agent Configuration: A Controlled Multi-Model Studycited by 0
  19. 2026A Blind Trust, the Bloody Thrust: When Attacker-Controlled Hook Updates Steer AI Agent Harnesses towards Malicious Behaviorscited by 0
  20. 2026ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agentscited by 0
  21. 2026Measuring and Exploiting Implicit Trust in LLM Tool-Calling Pipelinescited by 0
  22. 2026Connecting the Dots in Agentic AI Security: A Cross-Dimensional Threat Taxonomy, Evaluation Maturity, and Open Challengescited by 0
  23. 2026Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluationcited by 0
  24. 2026Persistent Billable State: Denial-of-Wallet Attacks and Defenses in Tool-Calling LLM Agentscited by 0
  25. 2026On the Effectiveness of Kernel-Level Evidence for Agent Securitycited by 0
  26. 2026PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agentscited by 0
  27. 2026COPEX: Benchmarking LLM Robustness to Adversarial Context Across Model Context Protocol Layerscited by 0
  28. 2026APEX: Active Protection at Execution Boundaries for LLM Agentscited by 0
  29. 2026Agent-Canary-Mas: End-to-End Evaluation of Protocol Risk in Long-Horizon Multi-Agent Systemscited by 0
  30. 2026AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safetycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents
  2. 2025MCPSecBench: A Systematic Security Benchmark and Playground for Testing Model Context Protocols
  3. 2025MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers
  4. 2025Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystem
  5. 2026Hybrid Analysis for Secure MCP Tool Use in LLM Agents
  6. 2025A First Look at the Security Issues in the Model Context Protocol Ecosystem
  7. 2025Parasites in the Toolchain: A Large-Scale Analysis of Attacks on the MCP Ecosystem
  8. 2026A Formal Security Framework for MCP-Based AI Agents: Threat Taxonomy, Verification Models, and Defense Mechanisms