mlsanityData to 7 Oct 2026

SoK: The Attack Surface of Agentic AI - Tools and Autonomy

arXiv 2603.22928 ↗First posted 24 Mar 2026cs.CRcited by 14

On the map: Retrieval-augmented generation → Privacy and security of RAG

also in 2 more
Abstract (arXiv)

Recent AI systems combine large language models with tools, external knowledge via retrieval-augmented generation (RAG), and even autonomous multi-agent decision loops. This agentic AI paradigm greatly expands capabilities - but also vastly enlarges the attack surface. In this systematization, we map out the trust boundaries and security risks of agentic LLM-based systems. We develop a comprehensive taxonomy of attacks spanning prompt-level injections, knowledge-base poisoning, tool/plug-in exploits, and multi-agent emergent threats. Through a detailed literature review, we synthesize evidence from 2023-2025, including more than 20 peer-reviewed and archival studies, industry reports, and standards. We find that agentic systems introduce new vectors for indirect prompt injection, code execution exploits, RAG index poisoning, and cross-agent manipulation that go beyond traditional AI threats. We define attacker models and threat scenarios, and propose metrics (e.g., Unsafe Action Rate, Privilege Escalation Distance) to evaluate security posture. Our survey examines defenses such as input sanitization, retrieval filters, sandboxes, access control, and "AI guardrails," assessing their effectiveness and pointing out the areas where protection is still lacking. To assist practitioners, we outline defensive controls and provide a phased security checklist for deploying agentic AI (covering design-time hardening, runtime monitoring, and incident response). Finally, we outline open research challenges in secure autonomous AI (robust tool APIs, verifiable agent behavior, supply-chain safeguards) and discuss ethical and responsible disclosure practices. We systematize recent findings to help researchers and engineers understand and mitigate security risks in agentic AI.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 16 of its 30 references that are in the map. First posted on arXiv 24 Mar 2026.

Builds on

16 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents9 Jun

Other references · 15

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  3. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 402
  4. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  5. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  6. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  7. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 224
  8. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
  9. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  10. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
  11. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  12. 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attackscited by 78
3 more
  1. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  2. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 36
  3. 2025Securing Agentic AI: A Comprehensive Threat Model and Mitigation Framework for Generative AI Agentscited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studiescited by 6
  2. 2026A Formal Security Framework for MCP-Based AI Agents: Threat Taxonomy, Verification Models, and Defense Mechanismscited by 2
  3. 2026LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environmentscited by 2
  4. 2026Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedbackcited by 2
  5. 2026A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problemscited by 0
  6. 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilitiescited by 0
  7. 2026LLM-Centric Agentic AI for UAV Swarms: Architecture, Enabling Technologies, and Open Problemscited by 0
  8. 2026From Agent Failure Paths to Quantified Residual Risk: A Compositional Framework for Resilient Agentic AIcited by 0
  9. 2026Where Is the Tradeoff in Using Third-Party API Routers for Agentic Software Development?cited by 0
  10. 2026Why LLM Agents Collapse Without Oversight: The Enforcement Gap as the Mechanism Behind Emergence World Failurescited by 0
  11. 2026Trust propagation and structural containment in Multi-agent LLM pipelinescited by 0
  12. 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systemscited by 0
2 more
  1. 2026Connecting the Dots in Agentic AI Security: A Cross-Dimensional Threat Taxonomy, Evaluation Maturity, and Open Challengescited by 0
  2. 2026Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework
  2. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
  3. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows
  4. 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks
  5. 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
  6. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
  7. 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise
  8. 2025Systems Security Foundations for Agentic Computing