A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
Agentic AI systems introduce a security surface that is qualitatively different from that of stateless LLMs. They persist memory, invoke external tools, coordinate with peer agents, and operate across sessions, allowing attacks to emerge not only at the prompt interface but also through architectural state, delegated authority, and long-horizon interactions. Existing security taxonomies, however, primarily organize threats by attack type, such as prompt injection or jailbreaking, and therefore obscure where in the agentic stack a threat arises and over what timescale it manifests. We propose the Layered Attack Surface Model (lasm), a structural taxonomy for agentic AI security. lasm decomposes the agentic stack into seven layers -- Foundation, Cognitive, Memory, Tool Execution, Multi-Agent Coordination, Ecosystem, and Governance -- and augments them with a four-class temporality axis covering instantaneous, session-persistent, cross-session cumulative, and sub-session-stack threats. We use this 7×4 framework to analyze 116 papers from 2021--2026. The resulting map reveals that the upper layers of the agentic stack remain sharply under-explored, especially for long-horizon and stack-propagating threats; multiple documented attack regions have no corresponding defenses; and current benchmarks provide no coverage for cross-session or sub-session-stack failure modes. We further derive a cross-layer defense taxonomy, defense recipes for canonical attack classes, and a dependency DAG that separates near-term engineering gaps from fundamental research challenges. We release the per-paper coding, robustness scripts, and a reference Agent Bill of Materials schema to support reproducible analysis.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 140 of its 180 references that are in the map. First posted on arXiv 25 Apr 2026.
Builds on
140 of 180 references.
Closest ancestors · 43
Same direction, newest first
- 2026Breaking MCP with Function Hijacking Attacks: Novel Threats for Function Calling and Agentic Models22 Apr
- 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle17 Apr
- 2026Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems3 Apr
- 2026Detecting Multi-Agent Collusion Through Multi-Agent Interpretability1 Apr
- 2026Agent Audit: A Security Analysis System for LLM Agent Applications24 Mar
- 2026From Storage to Steering: Memory Control Flow Attacks on LLM Agents16 Mar
- 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey11 Mar
- 2026Sleeper Cell: Injecting Latent Malice Temporal Backdoors into Tool-Using LLMs2 Mar
- 2026The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems19 Feb
- 2026"Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wild6 Feb
- 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems24 Jan
- 2026Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montage4 Jan
31 more
- 2025MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval18 Dec
- 2025MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers17 Dec
- 2025Systematization of Knowledge: Security and Safety in the Model Context Protocol Ecosystem9 Dec
- 2025AGENTSAFE: A Unified Framework for Ethical Assurance and Governance in Agentic AI2 Dec
- 2025Detecting Sleeper Agents in Large Language Models via Semantic Drift Analysis20 Nov
- 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges27 Oct
- 2025MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents14 Oct
- 2025Benchmarking the Robustness of Agentic Systems to Adversarially-Induced Harms22 Aug
- 2025MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers19 Aug
- 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety8 Jul
- 2025A Systematization of Security Vulnerabilities in Computer Use Agents7 Jul
- 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems4 Jun
- 2025Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents4 Apr
- 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasures12 Mar
- 2025Memory Injection Attacks on LLM Agents via Query-Only Interaction5 Mar
- 2025Unveiling Privacy Risks in LLM Agent Memory17 Feb
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification30 Jul
- 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies28 Jul
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
- 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents19 Jun
- 2024Dissecting Adversarial Robustness of Multimodal LM Agents18 Jun
- 2024Teams of LLM Agents can Exploit Zero-Day Vulnerabilities2 Jun
- 2024LLM Agents can Autonomously Exploit One-day Vulnerabilities11 Apr
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
- 2024WIPI: A New Web Threat for LLM-Driven Web Agents26 Feb
- 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
- 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganography12 Feb
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
- 2023Evil Geniuses: Delving into the Safety of LLM-based Agents20 Nov
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep
Other references · 97
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
38 more
- 2013Intriguing properties of neural networkscited by 5,114
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
- 2016Concrete Problems in AI Safetycited by 1,582
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
- 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
- 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
- 2022Red Teaming Language Models with Language Modelscited by 983
50 most cited of 97 All 97 (JSON)
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026Securing LLM-Agent Long-Term Memory Against Poisoning: Non-Malleable, Origin-Bound Authority with Machine-Checked Guaranteescited by 14
- 2026Toward Self-Evolution-Ready Workflow Harnesses: A Reversible Migration Path and Convertibility Taxonomy for Expert LLM Pipelinescited by 0
- 2026Agent Security Meets Regulatory Reality -- A Practitioner Systematization of Autonomous-Agent Threats and Controls in Regulated Financial Systemscited by 0
- 2026A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problemscited by 0
- 2026Workspace Topology as an Attack Vector in Agentic Coding Assistantscited by 0
- 2026Toward Secure AI-Powered Penetration Testing Agents: Security Threats, Guardrails, and Architectural Perspectivescited by 0
- 2026The Price of Safety: Benign-Case Utility and Token Overhead of Memory-Poisoning Defenses in LLM Agentscited by 0
- 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systemscited by 0
- 2026Reward Hacking and Agent Containment Failure: A Monte Carlo Study Based on the 2026 Hugging Face Incidentcited by 0
- 2026Reward Hacking and Agent Containment Failure: A Monte Carlo Study Based on the 2026 Hugging Face Incident26 Sep 2026
- 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems20 Sep 2026
- 2026The Price of Safety: Benign-Case Utility and Token Overhead of Memory-Poisoning Defenses in LLM Agents19 Sep 2026
- 2026Toward Secure AI-Powered Penetration Testing Agents: Security Threats, Guardrails, and Architectural Perspectives15 Sep 2026
- 2026Workspace Topology as an Attack Vector in Agentic Coding Assistants14 Aug 2026
- 2026A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problems30 Jun 2026
- 2026Agent Security Meets Regulatory Reality -- A Practitioner Systematization of Autonomous-Agent Threats and Controls in Regulated Financial Systems28 Jun 2026
- 2026Securing LLM-Agent Long-Term Memory Against Poisoning: Non-Malleable, Origin-Bound Authority with Machine-Checked Guarantees23 Jun 2026
- 2026Toward Self-Evolution-Ready Workflow Harnesses: A Reversible Migration Path and Convertibility Taxonomy for Expert LLM Pipelines15 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomy
- 2026Connecting the Dots in Agentic AI Security: A Cross-Dimensional Threat Taxonomy, Evaluation Maturity, and Open Challenges
- 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks
- 2025A Survey on Agentic Security: Applications, Threats and Defenses
- 2026Towards a Systems Foundation for Agentic Skills: Architecture, Lifecycle, and Security
- 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
- 2025Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation