mlsanityData to 7 Oct 2026

From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows

arXiv 2506.23260 ↗First posted 29 Jun 2025cs.CR cs.AIcited by 57

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Autonomous AI agents powered by large language models (LLMs) with structured function-calling interfaces enable real-time data retrieval, computation, and multi-step orchestration. However, the rapid growth of plugins, connectors, and inter-agent protocols has outpaced security practices, leading to brittle integrations that rely on ad-hoc authentication, inconsistent schemas, and weak validation. This survey introduces a unified end-to-end threat model for LLM-agent ecosystems, covering host-to-tool and agent-to-agent communications. We systematically categorize more than thirty attack techniques spanning input manipulation, model compromise, system and privacy attacks, and protocol-level vulnerabilities. For each category, we provide a formal threat formulation defining attacker capabilities, objectives, and affected system layers. Representative examples include Prompt-to-SQL injections and the Toxic Agent Flow exploit in GitHub MCP servers. We analyze attack feasibility, review existing defenses, and discuss mitigation strategies such as dynamic trust management, cryptographic provenance tracking, and sandboxed agent interfaces. The framework is validated through expert review and cross-mapping with real-world incidents and public vulnerability repositories, including CVE and NIST NVD. Compared to prior surveys, this work presents the first integrated taxonomy bridging input-level exploits and protocol-layer vulnerabilities in LLM-agent ecosystems, offering actionable guidance for designing secure and resilient agentic AI systems.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 51 of its 88 references that are in the map. First posted on arXiv 29 Jun 2025.

Builds on

51 of 88 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?16 May
  2. 2025A survey of agent interoperability protocols: Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent Protocol (A2A), and Agent Network Protocol (ANP)4 May
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr
  4. 2025MCP Bridge: A Lightweight, LLM-Agnostic RESTful Proxy for Model Context Protocol Servers11 Apr
  5. 2025Personalized Attacks of Social Engineering in Multi-turn Conversations: LLM Agents for Simulation and Detection18 Mar
  6. 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
  7. 2025CORBA: Contagious Recursive Blocking Attacks on Multi-Agent Systems Based on Large Language Models20 Feb
  8. 2025DemonAgent: Dynamically Encrypted Multi-Backdoor Implantation Attack on LLM-based Agent18 Feb
  9. 2024Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models1 Aug
  10. 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents5 Jun
  11. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents17 Feb

Other references · 40

  1. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  2. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  3. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  4. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  5. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  6. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 479
  7. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  8. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  9. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 260
  10. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
  11. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
  12. 2025Survey on Evaluation of LLM-based Agentscited by 190
28 more
  1. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 184
  2. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 167
  3. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  4. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  5. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 117
  6. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 106
  7. 2025A Survey of AI Agent Protocolscited by 77
  8. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76
  9. 2023A Pilot Study of Query-Free Adversarial Attack against Stable Diffusioncited by 75
  10. 2025Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Serverscited by 62
  11. 2022BadPrompt: Backdoor Attacks on Continuous Promptscited by 59
  12. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 58
  13. 2025Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systemscited by 58
  14. 2023From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?cited by 53
  15. 2025Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategiescited by 42
  16. 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Modelscited by 40
  17. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 39
  18. 2024Human-Imperceptible Retrieval Poisoning Attacks in LLM-Powered Applicationscited by 32
  19. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 29
  20. 2025Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agentscited by 23
  21. 2023FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMscited by 19
  22. 2025Build the web for agents, not agents for the webcited by 17
  23. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  24. 2025Graph of Attacks with Pruning: Optimizing Stealthy Jailbreak Prompt Generation for Enhanced LLM Content Moderationcited by 9
  25. 2025Surfer-H Meets Holo1: Cost-Efficient Web Agent Powered by Open Weightscited by 8
  26. 2025Agentic AI for Intent-Based Industrial Automationcited by 7
  27. 2025A Survey of Foundation Models for IoT: Taxonomy and Criteria-Based Analysiscited by 5
  28. 2025From Standalone LLMs to Integrated Intelligence: A Survey of Compound Al Systemscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 51.

57 papers.

Sort:

By area:

  1. 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystemscited by 30
  2. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  3. 2026ClawKeeper: Comprehensive Safety Protection for OpenClaw Agents Through Skills, Plugins, and Watcherscited by 18
  4. 2026SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skillscited by 18
  5. 2026A Framework for Formalizing LLM Agent Securitycited by 16
  6. 2026ClawSafety: "Safe" LLMs, Unsafe Agentscited by 15
  7. 2026From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaborationcited by 13
  8. 2025Semantic Attacks on Tool-Augmented LLMs: Securing the Model Context Protocol Against Descriptor-Level Manipulationcited by 12
  9. 2026Detecting Safety Violations Across Many Agent Tracescited by 11
  10. 2026Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Usecited by 9
  11. 2026SoK: Security of Autonomous LLM Agents in Agentic Commercecited by 8
  12. 2025Sentinel Agents for Secure and Trustworthy Agentic AI in Multi-Agent Systemscited by 7
38 more
  1. 2026SafeAgent: A Runtime Protection Architecture for Agentic Systemscited by 7
  2. 2025Security Analysis of Agentic AI Communication Protocols: A Comparative Evaluationcited by 6
  3. 2026When Bots Take the Bait: Exposing and Mitigating the Emerging Social Engineering Attack in Web Automation Agentcited by 6
  4. 2026MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detectioncited by 6
  5. 2026HDP: A Lightweight Cryptographic Protocol for Human Delegation Provenance in Agentic AI Systemscited by 5
  6. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 5
  7. 2025Trusted AI Agents in the Cloudcited by 4
  8. 2026Authenticated Workflows: A Systems Approach to Protecting Agentic AIcited by 4
  9. 2026Whispers of Wealth: Red-Teaming Google's Agent Payments Protocol via Prompt Injectioncited by 3
  10. 2026TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systemscited by 3
  11. 2026Enforcing Benign Trajectories: A Behavioral Firewall for Structured-Workflow AI Agentscited by 3
  12. 2026Trusted Credentials, Untrusted Behavior: Benchmarking LLM-Agent Security in High-Performance Computingcited by 3
  13. 2026A Formal Security Framework for MCP-Based AI Agents: Threat Taxonomy, Verification Models, and Defense Mechanismscited by 2
  14. 2026When Local Monitors Miss Compositional Harm: Diagnosing Distributed Backdoors in Multi-Agent Systemscited by 2
  15. 2025SME-TEAM: Leveraging Trust and Ethics for Secure and Responsible Use of AI and LLMs in SMEscited by 1
  16. 2026Buy versus Build an LLM: A Decision Framework for Governmentscited by 1
  17. 2026Security, privacy, and agentic AI in a regulatory view: From definitions and distinctions to provisions and reflectionscited by 1
  18. 2026Parallax: Why AI Agents That Think Must Never Actcited by 1
  19. 2026CyberAId: AI-Driven Cybersecurity for Financial Service Providerscited by 1
  20. 2026Trust Without Trusting: A Recomputable Trust Protocol for Autonomous Agentscited by 1
  21. 2026Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoningcited by 1
  22. 2026Attacks and Mitigations for Distributed Governance of Agentic AI under Byzantine Adversariescited by 1
  23. 2026"I Strongly Suspect This Website Is a Scam": Benchmarking PII Leakage and Detection without Defense in Autonomous Web Agentscited by 1
  24. 2026SGTO-MAS: Secure Gorilla Troops Optimization for Multi-Agent LLM Systemscited by 1
  25. 2026The Containment Gap: How Deployed Agentic AI Frameworks Fail Public-Facing Safety Requirementscited by 1
  26. 2026ChainWatch: A Kill Chain-Aligned Sequential Detection Framework for Multi-Step Attacks in MCP-Based AI Agent Systemscited by 1
  27. 2026IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requestscited by 1
  28. 2026Securing Agentic AI: From Per-Action Checks to Trajectory Assurancecited by 1
  29. 2026Blockchain Empowered Trustworthy Agent Networks: Foundations, Taxonomy, and Future Directionscited by 1
  30. 2026Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agentscited by 1
  31. 2026Security Attack and Defense Strategies for Autonomous Agent Frameworks: A Layered Review with OpenClaw as a Case Studycited by 0
  32. 2026Security Risks in Tool-Enabled AI Agents: A Systematic Analysis of Privileged Execution Environmentscited by 0
  33. 2026An Organization-Scoped LLM Agent Runtime Architecture for Regulated Cybersecurity Operationscited by 0
  34. 2026Certifiable Semantic Agreement Among LLM Agents: What the Admissibility Instrument Decidescited by 0
  35. 2026RIFT-Bench: Dynamic Red-teaming For Agentic AI Systemscited by 0
  36. 2026A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problemscited by 0
  37. 2026Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluationcited by 0
  38. 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Modelscited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigms
  2. 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomy
  3. 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise
  4. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
  5. 2025Design Patterns for Securing LLM Agents against Prompt Injections
  6. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems
  7. 2025A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks
  8. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis