The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
The evolution of Large Language Models (LLMs) has resulted in a paradigm shift towards autonomous agents, necessitating robust security against Prompt Injection (PI) vulnerabilities where untrusted inputs hijack agent behaviors. This SoK presents a comprehensive overview of the PI landscape, covering attacks, defenses, and their evaluation practices. Through a systematic literature review and quantitative analysis, we establish taxonomies that categorize PI attacks by payload generation strategies (heuristic vs. optimization) and defenses by intervention stages (text, model, and execution levels). Our analysis reveals a key limitation shared by many existing defenses and benchmarks: they largely overlook context-dependent tasks, in which agents are authorized to rely on runtime environmental observations to determine actions. To address this gap, we introduce AgentPI, a new benchmark designed to systematically evaluate agent behavior under context-dependent interaction settings. Using AgentPI, we empirically evaluate representative defenses and show that no single approach can simultaneously achieve high trustworthiness, high utility, and low latency. Moreover, we show that many defenses appear effective under existing benchmarks by suppressing contextual inputs, yet fail to generalize to realistic agent settings where context-dependent reasoning is essential. This SoK distills key takeaways and open research problems, offering structured guidance for future research and practical deployment of secure LLM agents.
Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 67 of its 96 references that are in the map. First posted on arXiv 11 Feb 2026.
Builds on
67 of 96 references.
Closest ancestors · 24
Same direction, newest first
- 2025Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs9 Dec
- 2025Exploiting Web Search Tools of AI Agents for Data Exfiltration10 Oct
- 2025CommandSans: Securing AI Agents with Surgical Precision Prompt Sanitization9 Oct
- 2025RL Is a Hammer and LLMs Are Nails: A Simple Reinforcement Learning Recipe for Strong Prompt Injection6 Oct
- 2025FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agents3 Oct
- 2025"Your AI, My Shell": Demystifying Prompt Injection Attacks on Agentic AI Coding Editors26 Sep
- 2025LLMZ+: Contextual Prompt Whitelist Principles for Agentic LLMs23 Sep
- 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection2 Aug
- 2025PromptArmor: Simple yet Effective Prompt Injection Defenses21 Jul
- 2025SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems9 Jun
- 2025To Protect the LLM Agent Against the Prompt Injection Attack with Polymorphic Prompt6 Jun
- 2025Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution1 Jun
12 more
- 2025Progent: Securing AI Agents with Privilege Control16 Apr
- 2025StruPhantom: Evolutionary Injection Attacks on Black-Box Tabular Agents Powered by Large Language Models14 Apr
- 2025Defeating Prompt Injections by Design24 Mar
- 2025Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents17 Mar
- 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
- 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection17 Feb
- 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage13 Feb
- 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents7 Feb
- 2024Towards Action Hijacking of Large Language Model-based Agent14 Dec
- 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems9 Oct
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
Other references · 43
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
- 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 255
- 2025Securing AI Agents with Information-Flow Controlcited by 121
- 2024Automatic and Universal Prompt Injection Attacks against Large Language Modelscited by 119
- 2024System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspectivecited by 96
- 2024Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applicationscited by 59
- 2025ACE: A Security Architecture for LLM-Integrated App Systemscited by 55
- 2025Can Indirect Prompt Injection Attacks Be Detected and Removed?cited by 52
- 2024Adversarial Search Engine Optimization for Large Language Modelscited by 40
- 2024Exploration of LLM Multi-Agent Application Implementation Based on LangGraph+CrewAIcited by 35
- 2025Defending Against Prompt Injection With a Few DefensiveTokenscited by 33
31 more
- 2025Defending against Indirect Prompt Injection by Instruction Detectioncited by 32
- 2025PromptLocate: Localizing Prompt Injection Attackscited by 28
- 2024HijackRAG: Hijacking Attacks against Retrieval-Augmented Large Language Modelscited by 27
- 2024Permissive Information-Flow Analysis for Large Language Modelscited by 22
- 2024FATH: Authentication-based Test-time Defense against Indirect Prompt Injection Attackscited by 22
- 2024Enhancing Prompt Injection Attacks to LLMs via Poisoning Alignmentcited by 21
- 2025PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Featurescited by 20
- 2024Preemptive Answer "Attacks" on Chain-of-Thought Reasoningcited by 19
- 2025When Your Reviewer is an LLM: Biases, Divergence, and Prompt Injection Risks in Peer Reviewcited by 19
- 2024PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMscited by 17
- 2025Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instructioncited by 16
- 2024Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattackscited by 15
- 2025May I have your Attention? Breaking Fine-Tuning based Prompt Injection Defenses using Architecture-Aware Attackscited by 15
- 2025Prompt Injection Attacks on LLM Generated Reviews of Scientific Publicationscited by 13
- 2025MAD-Spear: A Conformity-Driven Prompt Injection Attack on Multi-Agent Debate Systemscited by 10
- 2025SecInfer: Preventing Prompt Injection via Inference-time Scalingcited by 10
- 2024Prompt Injection Attacks on Large Language Models in Oncologycited by 8
- 2025A Taxonomy of Prompt Defects in LLM Systemscited by 8
- 2025Fingerprinting LLMs via Prompt Injectioncited by 8
- 2025VortexPIA: Indirect Prompt Injection Attack against LLMs for Efficient Extraction of User Privacycited by 7
- 2025Adversarial Attacks on LLM-as-a-Judge Systems: Insights from Prompt Injectionscited by 6
- 2025Misleading Large Language Models used (or misused) in Scientific Peer-Reviewing via Hidden Prompt-Injection Attackscited by 6
- 2024Scaling Behavior of Machine Translation with Large Language Models under Prompt Injection Attackscited by 4
- 2025Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning via Adaptive Token Compressioncited by 4
- 2025Too Easily Fooled? Prompt Injection Breaks LLMs on Frustratingly Simple Multiple-Choice Questionscited by 4
- 2025Counterfactual Evaluation for Blind Attack Detection in LLM-based Evaluation Systemscited by 3
- 2025AEGIS : Automated Co-Evolutionary Framework for Guarding Prompt Injections Schemacited by 3
- 2025A Call to Action for a Secure-by-Design Generative AI Paradigmcited by 3
- 2025Prompt Inject Detection with Generative Explanation as an Investigative Toolcited by 2
- 2025Breaking the Prompt Wall (I): A Real-World Case Study of Attacking ChatGPT via Lightweight Prompt Injectioncited by 2
- 2025Cybersecurity AI: Hacking the AI Hackers via Prompt Injectioncited by 2
Built on by
Citing papers per year: 2026: 30.
30 papers.
Sort:
By area:
- 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agentscited by 15
- 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injectioncited by 14
- 2026The Granularity Mismatch in Agent Security: Argument-Level Provenance Solves Enforcement and Isolates the LLM Reasoning Bottleneckcited by 11
- 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attackscited by 9
- 2026Reframing LLM Agent Security as an Agent-Human Interaction Problemcited by 9
- 2026SkillGuard: A Permission-Centric Framework for Agent Skill Securitycited by 6
- 2026AI Agents Under EU Lawcited by 5
- 2026No Attack Required: Semantic Fuzzing for Specification Violations in Agent Skillscited by 5
- 2026Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lenscited by 5
- 2026The Moltbook Observatory Archive: an incremental dataset of agent-only social network activitycited by 3
- 2026Trusted Credentials, Untrusted Behavior: Benchmarking LLM-Agent Security in High-Performance Computingcited by 3
- 2026AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrationscited by 2
18 more
- 2026Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agentscited by 2
- 2026AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectoriescited by 2
- 2026Constraining Host-Level Abuse in Self-Hosted Computer-Use Agents via TEE-Backed Isolationcited by 1
- 2026AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injectioncited by 1
- 2026LongPIBench: A Long-Context Benchmark for Prompt Injectioncited by 1
- 2026A Security Analysis of Long-Horizon Agentic AI Systems: Threats, Evaluation, and Framework Developmentcited by 0
- 2026Agent Security Meets Regulatory Reality -- A Practitioner Systematization of Autonomous-Agent Threats and Controls in Regulated Financial Systemscited by 0
- 2026Just Testing, Move Along: Evasion of LLM-based System Log Interpretation by Prompt Injectioncited by 0
- 2026Distributing Security Controls Through Harness Engineeringcited by 0
- 2026Workspace Topology as an Attack Vector in Agentic Coding Assistantscited by 0
- 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automationcited by 0
- 2026DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agentscited by 0
- 2026Beyond Single-Model Injection: A Threat Model and Defense Architecture for Prompt Injection in Multi-Agent Systemscited by 0
- 2026Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluationcited by 0
- 2026Persistent Billable State: Denial-of-Wallet Attacks and Defenses in Tool-Calling LLM Agentscited by 0
- 2026Pincer: Resource Authorization for Agents using a Digital Twincited by 0
- 2026RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agentscited by 0
- 2026Prompt-Injection Probes Learn the Conversation, Not Just the Injectioncited by 0
- 2026Prompt-Injection Probes Learn the Conversation, Not Just the Injection13 Oct 2026
- 2026RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents5 Oct 2026
- 2026Pincer: Resource Authorization for Agents using a Digital Twin1 Oct 2026
- 2026Persistent Billable State: Denial-of-Wallet Attacks and Defenses in Tool-Calling LLM Agents23 Sep 2026
- 2026Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluation21 Sep 2026
- 2026Beyond Single-Model Injection: A Threat Model and Defense Architecture for Prompt Injection in Multi-Agent Systems19 Sep 2026
- 2026DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agents9 Sep 2026
- 2026AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories7 Sep 2026
- 2026LongPIBench: A Long-Context Benchmark for Prompt Injection28 Aug 2026
- 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automation18 Aug 2026
- 2026Workspace Topology as an Attack Vector in Agentic Coding Assistants14 Aug 2026
- 2026AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injection4 Aug 2026
18 more
- 2026Distributing Security Controls Through Harness Engineering28 Jul 2026
- 2026Just Testing, Move Along: Evasion of LLM-based System Log Interpretation by Prompt Injection27 Jul 2026
- 2026Trusted Credentials, Untrusted Behavior: Benchmarking LLM-Agent Security in High-Performance Computing20 Jul 2026
- 2026Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens29 Jun 2026
- 2026Agent Security Meets Regulatory Reality -- A Practitioner Systematization of Autonomous-Agent Threats and Controls in Regulated Financial Systems28 Jun 2026
- 2026A Security Analysis of Long-Horizon Agentic AI Systems: Threats, Evaluation, and Framework Development12 Jun 2026
- 2026Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents11 Jun 2026
- 2026SkillGuard: A Permission-Centric Framework for Agent Skill Security2 Jun 2026
- 2026AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations1 Jun 2026
- 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agents26 May 2026
- 2026Reframing LLM Agent Security as an Agent-Human Interaction Problem23 May 2026
- 2026No Attack Required: Semantic Fuzzing for Specification Violations in Agent Skills13 May 2026
- 2026The Granularity Mismatch in Agent Security: Argument-Level Provenance Solves Enforcement and Isolates the LLM Reasoning Bottleneck11 May 2026
- 2026Constraining Host-Level Abuse in Self-Hosted Computer-Use Agents via TEE-Backed Isolation7 May 2026
- 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injection5 May 2026
- 2026The Moltbook Observatory Archive: an incremental dataset of agent-only social network activity16 Apr 2026
- 2026AI Agents Under EU Law6 Apr 2026
- 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks31 Mar 2026
- 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agentscited by 15
- 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injectioncited by 14
- 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attackscited by 9
- 2026Reframing LLM Agent Security as an Agent-Human Interaction Problemcited by 9
- 2026SkillGuard: A Permission-Centric Framework for Agent Skill Securitycited by 6
- 2026No Attack Required: Semantic Fuzzing for Specification Violations in Agent Skillscited by 5
- 2026Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lenscited by 5
- 2026Trusted Credentials, Untrusted Behavior: Benchmarking LLM-Agent Security in High-Performance Computingcited by 3
- 2026AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrationscited by 2
- 2026Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agentscited by 2
- 2026AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectoriescited by 2
- 2026Constraining Host-Level Abuse in Self-Hosted Computer-Use Agents via TEE-Backed Isolationcited by 1
12 most cited of 24 All 24 (JSON)
- 2026RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents5 Oct 2026
- 2026Pincer: Resource Authorization for Agents using a Digital Twin1 Oct 2026
- 2026Persistent Billable State: Denial-of-Wallet Attacks and Defenses in Tool-Calling LLM Agents23 Sep 2026
- 2026Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluation21 Sep 2026
- 2026Beyond Single-Model Injection: A Threat Model and Defense Architecture for Prompt Injection in Multi-Agent Systems19 Sep 2026
- 2026DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agents9 Sep 2026
- 2026AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories7 Sep 2026
- 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automation18 Aug 2026
- 2026Workspace Topology as an Attack Vector in Agentic Coding Assistants14 Aug 2026
- 2026AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injection4 Aug 2026
- 2026Distributing Security Controls Through Harness Engineering28 Jul 2026
- 2026Trusted Credentials, Untrusted Behavior: Benchmarking LLM-Agent Security in High-Performance Computing20 Jul 2026
12 newest of 24 All 24 (JSON)
- 2026The Moltbook Observatory Archive: an incremental dataset of agent-only social network activity16 Apr 2026
- 2026AI Agents Under EU Law6 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation
- 2026PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
- 2026Assessing Automated Prompt Injection Attacks in Agentic Environments
- 2026When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systems
- 2025Design Patterns for Securing LLM Agents against Prompt Injections
- 2025PromptSleuth: Detecting Prompt Injection via Semantic Intent Invariance
- 2025AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents
- 2025A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks