Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Large language model (LLM) agents are rapidly moving from conversational interfaces to software components that plan, invoke tools, maintain memory, and act on external environments. This transition changes the nature of security risk. In agentic settings, failures are no longer limited to unsafe text generation. Untrusted content may redirect control flow, misuse tool privileges, corrupt persistent state, leak sensitive information, or trigger harmful external actions. At the same time, research on LLM agent security is expanding quickly but remains fragmented across attack families, defense layers, application domains, and evaluation settings. This paper synthesizes 247 papers through a lifecycle-based, systems-oriented framework that models agent security around the interaction of information flow, delegated authority, and persistent state. We organize the literature around four questions: how LLM agent security should be modeled, which threat surfaces and attack families dominate, what defenses have been proposed and with what tradeoffs, and how security claims are evaluated. We find that prompt injection and tool-mediated control-flow hijacking still dominate the field, while persistent state corruption and multi-agent propagation are becoming central emerging concerns. We further find that current defenses provide useful building blocks but remain weakly compositional, and that existing benchmarks still underrepresent long-horizon, stateful, and deployment-sensitive risks. We argue that secure LLM agents require explicit trust boundaries, principled privilege control, provenance-aware state management, and evaluation practices aligned with realistic operational settings.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 197 of its 244 references that are in the map. First posted on arXiv 9 Jun 2026.
Builds on
197 of 244 references.
Closest ancestors · 159
Same direction, newest first
- 2026CIA: Inferring the Communication Topology from LLM-based Multi-Agent Systems14 Apr
- 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection13 Apr
- 2026PlanGuard: Defending Agents against Indirect Prompt Injection via Planning-based Consistency Verification11 Apr
- 2026Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain9 Apr
- 2026Semantic Intent Fragmentation: A Single-Shot Compositional Attack on Multi-Agent AI Pipelines8 Apr
- 2026SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems8 Apr
- 2026Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use7 Apr
- 2026SkillAttack: Automated Red Teaming of Agent Skills through Attack Path Refinement5 Apr
- 2026Your Agent is More Brittle Than You Think: Uncovering Indirect Injection Vulnerabilities in Agentic LLMs4 Apr
- 2026Towards Secure Agent Skills: Architecture, Threat Taxonomy, and Security Analysis3 Apr
- 2026ClawSafety: "Safe" LLMs, Unsafe Agents1 Apr
- 2026No Attacker Needed: Unintentional Cross-User Contamination in Shared-State LLM Agents1 Apr
38 more
- 2026A Framework for Formalizing LLM Agent Security19 Mar
- 2026Towards Reliable Local Security Agents: Verifiable Post-Training for Linux Privilege Escalation18 Mar
- 2026How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition16 Mar
- 2026From Storage to Steering: Memory Control Flow Attacks on LLM Agents16 Mar
- 2026OpenClaw PRISM: A Zero-Fork, Defense-in-Depth Runtime Security Layer for Tool-Augmented LLM Agents12 Mar
- 2026Taming OpenClaw: Security Analysis and Mitigation of Autonomous LLM Agent Threats12 Mar
- 2026WebWeaver: Breaking Topology Confidentiality in LLM Multi-Agent Systems with Stealthy Context-Based Inference11 Mar
- 2026AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocations11 Mar
- 2026AgenticCyOps: Securing Multi-Agentic AI Integration in Enterprise Cyber Operations10 Mar
- 2026From Thinker to Society: Security in Hierarchical Autonomy Evolution of AI Agents8 Mar
- 2026Auditing Cascading Risks in Multi-Agent Systems via Semantic-Geometric Co-evolution4 Mar
- 2026Cross-Layer Semantic Flow Reconstruction for Attack Detection in Agentic Systems4 Mar
- 2026AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification26 Feb
- 2026Silent Egress: When Implicit Prompt Injection Makes LLM Agents Leak Without a Trace25 Feb
- 2026AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs24 Feb
- 2026ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correction24 Feb
- 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks23 Feb
- 2026SOK: A Taxonomy of Attack Vectors and Defense Strategies for Agentic Supply Chain Runtime23 Feb
- 2026AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks18 Feb
- 2026Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections17 Feb
- 2026Overthinking Loops in Agents: A Structural Risk via MCP Tools16 Feb
- 2026SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents15 Feb
- 2026Information-Theoretic Privacy Control for Sequential Multi-Agent LLM Systems13 Feb
- 2026OMNI-LEAK: Orchestrator Multi-Agent Network Induced Data Leakage13 Feb
- 2026MalTool: Malicious Tool Attacks on LLM Agents12 Feb
- 2026Authenticated Workflows: A Systems Approach to Protecting Agentic AI11 Feb
- 2026From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agent9 Feb
- 2026AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Management7 Feb
- 2026Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations5 Feb
- 2026Bypassing AI Control Protocols via Agent-as-a-Proxy Attacks4 Feb
- 2026Human Society-Inspired Approaches to Agentic AI Security: The 4C Framework2 Feb
- 2026Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents24 Jan
- 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems24 Jan
- 2026AgenTRIM: Tool Risk Mitigation for Agentic AI18 Jan
- 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework17 Jan
- 2026Beyond Max Tokens: Stealthy Resource Amplification via Tool Calling Chains in LLM Agents16 Jan
- 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback15 Jan
- 2026Agents at Risk: How Users Unwittingly Undermine LLM Safety14 Jan
50 newest of 159 All 159 (JSON)
Other references · 38
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
- 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
- 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
- 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
- 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agentscited by 120
26 more
- 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 105
- 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 77
- 2025ACE: A Security Architecture for LLM-Integrated App Systemscited by 52
- 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 49
- 2025CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactionscited by 46
- 2026AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?cited by 45
- 2025Lessons from Defending Gemini Against Indirect Prompt Injectionscited by 40
- 2025Towards Trustworthy GUI Agents: A Surveycited by 33
- 2025MCIP: Protecting MCP Safety via Model Contextual Integrity Protocolcited by 33
- 2024Exploring the Robustness of Decision-Level Through Adversarial Attacks on LLM-Based Embodied Modelscited by 31
- 2025The Traitors: Deception and Trust in Multi-Agent Language Model Simulationscited by 30
- 2025OS-Kairos: Adaptive Interaction for MLLM-Powered GUI Agentscited by 25
- 2025Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agentscited by 23
- 2025LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challengecited by 20
- 2025When MCP Servers Attack: Taxonomy, Feasibility, and Mitigationcited by 17
- 2025Forewarned is Forearmed: A Survey on Large Language Model-based Agents in Autonomous Cyberattackscited by 15
- 2024Easier Said Than Done: Unpacking Intent-Behavior Gap in Jailbreaking LLM-Based Robotscited by 14
- 2025Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerouscited by 11
- 2024A Troublemaker with Contagious Jailbreak Makes Chaos in Honest Townscited by 9
- 2026What Breaks Embodied AI Security:LLM Vulnerabilities, CPS Flaws,or Something Else?cited by 5
- 2026ACIArena: Toward Unified Evaluation for Agent Cascading Injectioncited by 5
- 2025The Shawshank Redemption of Embodied AI: Understanding and Benchmarking Indirect Environmental Jailbreakscited by 4
- 2026Systems-Level Attack Surface of Edge Agent Deployments on IoTcited by 3
- 2026Are AI-assisted Development Tools Immune to Prompt Injection?cited by 3
- 2026AttackEval: A Systematic Empirical Study of Prompt Injection Attack Effectiveness Against Large Language Modelscited by 2
- 2026CONSCIENTIA: Can LLM Agents Learn to Strategize? Emergent Deception and Trust in a Multi-Agent NYC Simulationcited by 2
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026CAGE: Certified Authorization under Typed-Return Uncertainty for Tool-Using Agentscited by 1
- 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agentscited by 0
- 2026Security of Foundation-Model-Powered Embodied Agents: Attack Surfaces, Attacks, Defenses, and Evaluationcited by 0
- 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systemscited by 0
- 2026Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluationcited by 0
- 2026Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluation25 Sep 2026
- 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems20 Sep 2026
- 2026Security of Foundation-Model-Powered Embodied Agents: Attack Surfaces, Attacks, Defenses, and Evaluation17 Aug 2026
- 2026CAGE: Certified Authorization under Typed-Return Uncertainty for Tool-Using Agents31 Jul 2026
- 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agents25 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models
- 2024Whispers in the Machine: Confidentiality in Agentic Systems
- 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise
- 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
- 2026LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment
- 2026From Secure Agentic AI to Secure Agentic Web: Challenges, Threats, and Future Directions
- 2025A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks
- 2025Security Concerns for Large Language Models: A Survey