mlsanityData to 7 Oct 2026

Invisible Threats from Model Context Protocol: Generating Stealthy Injection Payload via Tree-based Adaptive Search

arXiv 2603.24203 ↗First posted 25 Mar 2026cs.CR cs.AIcited by 2

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Recent advances in the Model Context Protocol (MCP) have enabled large language models (LLMs) to invoke external tools with unprecedented ease. This creates a new class of powerful and tool augmented agents. Unfortunately, this capability also introduces an under explored attack surface, specifically the malicious manipulation of tool responses. Existing techniques for indirect prompt injection that target MCP suffer from high deployment costs, weak semantic coherence, or heavy white box requirements. Furthermore, they are often easily detected by recently proposed defenses. In this paper, we propose Tree structured Injection for Payloads (TIP), a novel black-box attack which generates natural payloads to reliably seize control of MCP enabled agents even under defense. Technically, We cast payload generation as a tree structured search problem and guide the search with an attacker LLM operating under our proposed coarse-to-fine optimization framework. To stabilize learning and avoid local optima, we introduce a path-aware feedback mechanism that surfaces only high quality historical trajectories to the attacker model. The framework is further hardened against defensive transformations by explicitly conditioning the search on observable defense signals and dynamically reallocating the exploration budget. Extensive experiments on four mainstream LLMs show that TIP attains over 95% attack success in undefended settings while requiring an order of magnitude fewer queries than prior adaptive attacks. Against four representative defense approaches, TIP preserves more than 50% effectiveness and significantly outperforms the state-of-the-art attacks. By implementing the attack on real world MCP systems, our results expose an invisible but practical threat vector in MCP deployments. We also discuss potential mitigation approaches to address this critical security gap.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 47 of its 57 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

47 of 57 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents14 Oct
  2. 2025Parasites in the Toolchain: A Large-Scale Analysis of Attacks on the MCP Ecosystem8 Sep
  3. 2025MCPXKIT: The Unified Toolkit for Analyzing Model Context Protocol Security18 Aug
  4. 2025We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems16 Jun
  5. 2025MPMA: Preference Manipulation Attack Against Model Context Protocol16 May
  6. 2025MCP Guardian: A Security-First Layer for Safeguarding MCP-Based AI System17 Apr
  7. 2025UDora: A Unified Red Teaming Framework against LLM Agents by Dynamically Hijacking Their Own Reasoning28 Feb
  8. 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
  9. 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage13 Feb
  10. 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents21 Dec
  11. 2024From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection13 Dec
  12. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
3 more
  1. 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification30 Jul
  2. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  3. 2023Evil Geniuses: Delving into the Safety of LLM-based Agents20 Nov

Other references · 32

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  9. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  10. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  11. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  12. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
20 more
  1. 2024Qwen2 Technical Reportcited by 2,076
  2. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  3. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  4. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  5. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  6. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  7. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  8. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  9. 2023Detecting Language Model Attacks with Perplexitycited by 411
  10. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  11. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
  12. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  13. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
  14. 2024Tool Learning with Large Language Models: A Surveycited by 221
  15. 2016Dual Learning for Machine Translationcited by 146
  16. 2024Automatic and Universal Prompt Injection Attacks against Large Language Modelscited by 112
  17. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Modelscited by 102
  18. 2025Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategiescited by 42
  19. 2023Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial Attackscited by 32
  20. 2025When MCP Servers Attack: Taxonomy, Feasibility, and Mitigationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026MCP-DPT: A Defense-Placement Taxonomy and Coverage Analysis for Model Context Protocol Security
  2. 2025MCPSecBench: A Systematic Security Benchmark and Playground for Testing Model Context Protocols
  3. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
  4. 2026Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents
  5. 2025Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystem
  6. 2026A Formal Security Framework for MCP-Based AI Agents: Threat Taxonomy, Verification Models, and Defense Mechanisms
  7. 2025MCP-Guard: A Multi-Stage Defense-in-Depth Framework for Securing Model Context Protocol in Agentic AI
  8. 2025IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents