mlsanityData to 14 Oct 2026

The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis

arXiv 2602.10453 ↗First posted 11 Feb 2026cs.CR cs.CLcited by 30

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

The evolution of Large Language Models (LLMs) has resulted in a paradigm shift towards autonomous agents, necessitating robust security against Prompt Injection (PI) vulnerabilities where untrusted inputs hijack agent behaviors. This SoK presents a comprehensive overview of the PI landscape, covering attacks, defenses, and their evaluation practices. Through a systematic literature review and quantitative analysis, we establish taxonomies that categorize PI attacks by payload generation strategies (heuristic vs. optimization) and defenses by intervention stages (text, model, and execution levels). Our analysis reveals a key limitation shared by many existing defenses and benchmarks: they largely overlook context-dependent tasks, in which agents are authorized to rely on runtime environmental observations to determine actions. To address this gap, we introduce AgentPI, a new benchmark designed to systematically evaluate agent behavior under context-dependent interaction settings. Using AgentPI, we empirically evaluate representative defenses and show that no single approach can simultaneously achieve high trustworthiness, high utility, and low latency. Moreover, we show that many defenses appear effective under existing benchmarks by suppressing contextual inputs, yet fail to generalize to realistic agent settings where context-dependent reasoning is essential. This SoK distills key takeaways and open research problems, offering structured guidance for future research and practical deployment of secure LLM agents.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 67 of its 96 references that are in the map. First posted on arXiv 11 Feb 2026.

Builds on

67 of 96 references.

Closest ancestors · 24

Same direction, newest first

  1. 2025Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs9 Dec
  2. 2025Exploiting Web Search Tools of AI Agents for Data Exfiltration10 Oct
  3. 2025CommandSans: Securing AI Agents with Surgical Precision Prompt Sanitization9 Oct
  4. 2025RL Is a Hammer and LLMs Are Nails: A Simple Reinforcement Learning Recipe for Strong Prompt Injection6 Oct
  5. 2025FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agents3 Oct
  6. 2025"Your AI, My Shell": Demystifying Prompt Injection Attacks on Agentic AI Coding Editors26 Sep
  7. 2025LLMZ+: Contextual Prompt Whitelist Principles for Agentic LLMs23 Sep
  8. 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection2 Aug
  9. 2025PromptArmor: Simple yet Effective Prompt Injection Defenses21 Jul
  10. 2025SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems9 Jun
  11. 2025To Protect the LLM Agent Against the Prompt Injection Attack with Polymorphic Prompt6 Jun
  12. 2025Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution1 Jun
12 more
  1. 2025Progent: Securing AI Agents with Privilege Control16 Apr
  2. 2025StruPhantom: Evolutionary Injection Attacks on Black-Box Tabular Agents Powered by Large Language Models14 Apr
  3. 2025Defeating Prompt Injections by Design24 Mar
  4. 2025Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents17 Mar
  5. 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
  6. 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection17 Feb
  7. 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage13 Feb
  8. 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents7 Feb
  9. 2024Towards Action Hijacking of Large Language Model-based Agent14 Dec
  10. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems9 Oct
  11. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  12. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar

Other references · 43

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  2. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
  3. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 255
  4. 2025Securing AI Agents with Information-Flow Controlcited by 121
  5. 2024Automatic and Universal Prompt Injection Attacks against Large Language Modelscited by 119
  6. 2024System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspectivecited by 96
  7. 2024Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applicationscited by 59
  8. 2025ACE: A Security Architecture for LLM-Integrated App Systemscited by 55
  9. 2025Can Indirect Prompt Injection Attacks Be Detected and Removed?cited by 52
  10. 2024Adversarial Search Engine Optimization for Large Language Modelscited by 40
  11. 2024Exploration of LLM Multi-Agent Application Implementation Based on LangGraph+CrewAIcited by 35
  12. 2025Defending Against Prompt Injection With a Few DefensiveTokenscited by 33
31 more
  1. 2025Defending against Indirect Prompt Injection by Instruction Detectioncited by 32
  2. 2025PromptLocate: Localizing Prompt Injection Attackscited by 28
  3. 2024HijackRAG: Hijacking Attacks against Retrieval-Augmented Large Language Modelscited by 27
  4. 2024Permissive Information-Flow Analysis for Large Language Modelscited by 22
  5. 2024FATH: Authentication-based Test-time Defense against Indirect Prompt Injection Attackscited by 22
  6. 2024Enhancing Prompt Injection Attacks to LLMs via Poisoning Alignmentcited by 21
  7. 2025PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Featurescited by 20
  8. 2024Preemptive Answer "Attacks" on Chain-of-Thought Reasoningcited by 19
  9. 2025When Your Reviewer is an LLM: Biases, Divergence, and Prompt Injection Risks in Peer Reviewcited by 19
  10. 2024PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMscited by 17
  11. 2025Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instructioncited by 16
  12. 2024Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattackscited by 15
  13. 2025May I have your Attention? Breaking Fine-Tuning based Prompt Injection Defenses using Architecture-Aware Attackscited by 15
  14. 2025Prompt Injection Attacks on LLM Generated Reviews of Scientific Publicationscited by 13
  15. 2025MAD-Spear: A Conformity-Driven Prompt Injection Attack on Multi-Agent Debate Systemscited by 10
  16. 2025SecInfer: Preventing Prompt Injection via Inference-time Scalingcited by 10
  17. 2024Prompt Injection Attacks on Large Language Models in Oncologycited by 8
  18. 2025A Taxonomy of Prompt Defects in LLM Systemscited by 8
  19. 2025Fingerprinting LLMs via Prompt Injectioncited by 8
  20. 2025VortexPIA: Indirect Prompt Injection Attack against LLMs for Efficient Extraction of User Privacycited by 7
  21. 2025Adversarial Attacks on LLM-as-a-Judge Systems: Insights from Prompt Injectionscited by 6
  22. 2025Misleading Large Language Models used (or misused) in Scientific Peer-Reviewing via Hidden Prompt-Injection Attackscited by 6
  23. 2024Scaling Behavior of Machine Translation with Large Language Models under Prompt Injection Attackscited by 4
  24. 2025Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning via Adaptive Token Compressioncited by 4
  25. 2025Too Easily Fooled? Prompt Injection Breaks LLMs on Frustratingly Simple Multiple-Choice Questionscited by 4
  26. 2025Counterfactual Evaluation for Blind Attack Detection in LLM-based Evaluation Systemscited by 3
  27. 2025AEGIS : Automated Co-Evolutionary Framework for Guarding Prompt Injections Schemacited by 3
  28. 2025A Call to Action for a Secure-by-Design Generative AI Paradigmcited by 3
  29. 2025Prompt Inject Detection with Generative Explanation as an Investigative Toolcited by 2
  30. 2025Breaking the Prompt Wall (I): A Real-World Case Study of Attacking ChatGPT via Lightweight Prompt Injectioncited by 2
  31. 2025Cybersecurity AI: Hacking the AI Hackers via Prompt Injectioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 30.

30 papers.

Sort:

By area:

  1. 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agentscited by 15
  2. 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injectioncited by 14
  3. 2026The Granularity Mismatch in Agent Security: Argument-Level Provenance Solves Enforcement and Isolates the LLM Reasoning Bottleneckcited by 11
  4. 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attackscited by 9
  5. 2026Reframing LLM Agent Security as an Agent-Human Interaction Problemcited by 9
  6. 2026SkillGuard: A Permission-Centric Framework for Agent Skill Securitycited by 6
  7. 2026AI Agents Under EU Lawcited by 5
  8. 2026No Attack Required: Semantic Fuzzing for Specification Violations in Agent Skillscited by 5
  9. 2026Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lenscited by 5
  10. 2026The Moltbook Observatory Archive: an incremental dataset of agent-only social network activitycited by 3
  11. 2026Trusted Credentials, Untrusted Behavior: Benchmarking LLM-Agent Security in High-Performance Computingcited by 3
  12. 2026AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrationscited by 2
18 more
  1. 2026Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agentscited by 2
  2. 2026AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectoriescited by 2
  3. 2026Constraining Host-Level Abuse in Self-Hosted Computer-Use Agents via TEE-Backed Isolationcited by 1
  4. 2026AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injectioncited by 1
  5. 2026LongPIBench: A Long-Context Benchmark for Prompt Injectioncited by 1
  6. 2026A Security Analysis of Long-Horizon Agentic AI Systems: Threats, Evaluation, and Framework Developmentcited by 0
  7. 2026Agent Security Meets Regulatory Reality -- A Practitioner Systematization of Autonomous-Agent Threats and Controls in Regulated Financial Systemscited by 0
  8. 2026Just Testing, Move Along: Evasion of LLM-based System Log Interpretation by Prompt Injectioncited by 0
  9. 2026Distributing Security Controls Through Harness Engineeringcited by 0
  10. 2026Workspace Topology as an Attack Vector in Agentic Coding Assistantscited by 0
  11. 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automationcited by 0
  12. 2026DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agentscited by 0
  13. 2026Beyond Single-Model Injection: A Threat Model and Defense Architecture for Prompt Injection in Multi-Agent Systemscited by 0
  14. 2026Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluationcited by 0
  15. 2026Persistent Billable State: Denial-of-Wallet Attacks and Defenses in Tool-Calling LLM Agentscited by 0
  16. 2026Pincer: Resource Authorization for Agents using a Digital Twincited by 0
  17. 2026RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agentscited by 0
  18. 2026Prompt-Injection Probes Learn the Conversation, Not Just the Injectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation
  2. 2026PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
  3. 2026Assessing Automated Prompt Injection Attacks in Agentic Environments
  4. 2026When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systems
  5. 2025Design Patterns for Securing LLM Agents against Prompt Injections
  6. 2025PromptSleuth: Detecting Prompt Injection via Semantic Intent Invariance
  7. 2025AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents
  8. 2025A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks