mlsanityData to 7 Oct 2026

Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation

arXiv 2606.10749 ↗First posted 9 Jun 2026cs.CR cs.AIcited by 5

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Large language model (LLM) agents are rapidly moving from conversational interfaces to software components that plan, invoke tools, maintain memory, and act on external environments. This transition changes the nature of security risk. In agentic settings, failures are no longer limited to unsafe text generation. Untrusted content may redirect control flow, misuse tool privileges, corrupt persistent state, leak sensitive information, or trigger harmful external actions. At the same time, research on LLM agent security is expanding quickly but remains fragmented across attack families, defense layers, application domains, and evaluation settings. This paper synthesizes 247 papers through a lifecycle-based, systems-oriented framework that models agent security around the interaction of information flow, delegated authority, and persistent state. We organize the literature around four questions: how LLM agent security should be modeled, which threat surfaces and attack families dominate, what defenses have been proposed and with what tradeoffs, and how security claims are evaluated. We find that prompt injection and tool-mediated control-flow hijacking still dominate the field, while persistent state corruption and multi-agent propagation are becoming central emerging concerns. We further find that current defenses provide useful building blocks but remain weakly compositional, and that existing benchmarks still underrepresent long-horizon, stateful, and deployment-sensitive risks. We argue that secure LLM agents require explicit trust boundaries, principled privilege control, provenance-aware state management, and evaluation practices aligned with realistic operational settings.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 197 of its 244 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

197 of 244 references.

Closest ancestors · 159

Same direction, newest first

  1. 2026CIA: Inferring the Communication Topology from LLM-based Multi-Agent Systems14 Apr
  2. 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection13 Apr
  3. 2026PlanGuard: Defending Agents against Indirect Prompt Injection via Planning-based Consistency Verification11 Apr
  4. 2026Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain9 Apr
  5. 2026Semantic Intent Fragmentation: A Single-Shot Compositional Attack on Multi-Agent AI Pipelines8 Apr
  6. 2026SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems8 Apr
  7. 2026Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use7 Apr
  8. 2026SkillAttack: Automated Red Teaming of Agent Skills through Attack Path Refinement5 Apr
  9. 2026Your Agent is More Brittle Than You Think: Uncovering Indirect Injection Vulnerabilities in Agentic LLMs4 Apr
  10. 2026Towards Secure Agent Skills: Architecture, Threat Taxonomy, and Security Analysis3 Apr
  11. 2026ClawSafety: "Safe" LLMs, Unsafe Agents1 Apr
  12. 2026No Attacker Needed: Unintentional Cross-User Contamination in Shared-State LLM Agents1 Apr
38 more
  1. 2026A Framework for Formalizing LLM Agent Security19 Mar
  2. 2026Towards Reliable Local Security Agents: Verifiable Post-Training for Linux Privilege Escalation18 Mar
  3. 2026How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition16 Mar
  4. 2026From Storage to Steering: Memory Control Flow Attacks on LLM Agents16 Mar
  5. 2026OpenClaw PRISM: A Zero-Fork, Defense-in-Depth Runtime Security Layer for Tool-Augmented LLM Agents12 Mar
  6. 2026Taming OpenClaw: Security Analysis and Mitigation of Autonomous LLM Agent Threats12 Mar
  7. 2026WebWeaver: Breaking Topology Confidentiality in LLM Multi-Agent Systems with Stealthy Context-Based Inference11 Mar
  8. 2026AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocations11 Mar
  9. 2026AgenticCyOps: Securing Multi-Agentic AI Integration in Enterprise Cyber Operations10 Mar
  10. 2026From Thinker to Society: Security in Hierarchical Autonomy Evolution of AI Agents8 Mar
  11. 2026Auditing Cascading Risks in Multi-Agent Systems via Semantic-Geometric Co-evolution4 Mar
  12. 2026Cross-Layer Semantic Flow Reconstruction for Attack Detection in Agentic Systems4 Mar
  13. 2026AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification26 Feb
  14. 2026Silent Egress: When Implicit Prompt Injection Makes LLM Agents Leak Without a Trace25 Feb
  15. 2026AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs24 Feb
  16. 2026ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correction24 Feb
  17. 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks23 Feb
  18. 2026SOK: A Taxonomy of Attack Vectors and Defense Strategies for Agentic Supply Chain Runtime23 Feb
  19. 2026AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks18 Feb
  20. 2026Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections17 Feb
  21. 2026Overthinking Loops in Agents: A Structural Risk via MCP Tools16 Feb
  22. 2026SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents15 Feb
  23. 2026Information-Theoretic Privacy Control for Sequential Multi-Agent LLM Systems13 Feb
  24. 2026OMNI-LEAK: Orchestrator Multi-Agent Network Induced Data Leakage13 Feb
  25. 2026MalTool: Malicious Tool Attacks on LLM Agents12 Feb
  26. 2026Authenticated Workflows: A Systems Approach to Protecting Agentic AI11 Feb
  27. 2026From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agent9 Feb
  28. 2026AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Management7 Feb
  29. 2026Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations5 Feb
  30. 2026Bypassing AI Control Protocols via Agent-as-a-Proxy Attacks4 Feb
  31. 2026Human Society-Inspired Approaches to Agentic AI Security: The 4C Framework2 Feb
  32. 2026Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents24 Jan
  33. 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems24 Jan
  34. 2026AgenTRIM: Tool Risk Mitigation for Agentic AI18 Jan
  35. 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework17 Jan
  36. 2026Beyond Max Tokens: Stealthy Resource Amplification via Tool Calling Chains in LLM Agents16 Jan
  37. 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback15 Jan
  38. 2026Agents at Risk: How Users Unwittingly Undermine LLM Safety14 Jan

50 newest of 159 All 159 (JSON)

Other references · 38

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  6. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  7. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  8. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  9. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
  10. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
  11. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  12. 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agentscited by 120
26 more
  1. 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 105
  2. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 77
  3. 2025ACE: A Security Architecture for LLM-Integrated App Systemscited by 52
  4. 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 49
  5. 2025CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactionscited by 46
  6. 2026AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?cited by 45
  7. 2025Lessons from Defending Gemini Against Indirect Prompt Injectionscited by 40
  8. 2025Towards Trustworthy GUI Agents: A Surveycited by 33
  9. 2025MCIP: Protecting MCP Safety via Model Contextual Integrity Protocolcited by 33
  10. 2024Exploring the Robustness of Decision-Level Through Adversarial Attacks on LLM-Based Embodied Modelscited by 31
  11. 2025The Traitors: Deception and Trust in Multi-Agent Language Model Simulationscited by 30
  12. 2025OS-Kairos: Adaptive Interaction for MLLM-Powered GUI Agentscited by 25
  13. 2025Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agentscited by 23
  14. 2025LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challengecited by 20
  15. 2025When MCP Servers Attack: Taxonomy, Feasibility, and Mitigationcited by 17
  16. 2025Forewarned is Forearmed: A Survey on Large Language Model-based Agents in Autonomous Cyberattackscited by 15
  17. 2024Easier Said Than Done: Unpacking Intent-Behavior Gap in Jailbreaking LLM-Based Robotscited by 14
  18. 2025Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerouscited by 11
  19. 2024A Troublemaker with Contagious Jailbreak Makes Chaos in Honest Townscited by 9
  20. 2026What Breaks Embodied AI Security:LLM Vulnerabilities, CPS Flaws,or Something Else?cited by 5
  21. 2026ACIArena: Toward Unified Evaluation for Agent Cascading Injectioncited by 5
  22. 2025The Shawshank Redemption of Embodied AI: Understanding and Benchmarking Indirect Environmental Jailbreakscited by 4
  23. 2026Systems-Level Attack Surface of Edge Agent Deployments on IoTcited by 3
  24. 2026Are AI-assisted Development Tools Immune to Prompt Injection?cited by 3
  25. 2026AttackEval: A Systematic Empirical Study of Prompt Injection Attack Effectiveness Against Large Language Modelscited by 2
  26. 2026CONSCIENTIA: Can LLM Agents Learn to Strategize? Emergent Deception and Trust in a Multi-Agent NYC Simulationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models
  2. 2024Whispers in the Machine: Confidentiality in Agentic Systems
  3. 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise
  4. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
  5. 2026LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment
  6. 2026From Secure Agentic AI to Secure Agentic Web: Challenges, Threats, and Future Directions
  7. 2025A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks
  8. 2025Security Concerns for Large Language Models: A Survey