AI Agents May Always Fall for Prompt Injections
On the map: LLM safety and security → Prompt injection and agent security
Abstract (arXiv)
Prompt injection is the most critical vulnerability in deployed AI agents. Despite recent progress, we show that the prevailing defense paradigm (data-instruction separation) both fails to detect attacks that operate through contextual manipulation and degrades contextually appropriate behavior. We then recast prompt injection via the lens of Contextual Integrity (CI), a privacy theory that judges information flow compliance with contextual norms. This explains types of attacks that current defenses attempt to patch and predict advanced ones future agents will face. We develop unique benign and attack scenarios that force an agent to violate the norms by (1) misrepresenting the flow, (2) manipulating norms, or (3) mixing multiple flows. This reframing suggests an impossibility result: an adversary can always construct a context under which a blocked flow appears legitimate, or a defender who tightens norms will block genuinely legitimate flows. Our findings suggest that current research addresses a shrinking fraction of future attack surfaces. Instead, through CI, we offer a principled framework for evaluating context-sensitive failures, and designing CI-aware alignment for the frontier autonomous agents.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 22 of its 37 references that are in the map. First posted on arXiv 17 May 2026.
Builds on
22 of 37 references.
Closest ancestors · 8
Same direction, newest first
- 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks23 Feb
- 2026Learning to Inject: Automated Prompt Injection via Reinforcement Learning5 Feb
- 2025Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols10 Oct
- 2025LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge11 Jun
- 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks15 Apr
- 2025Defeating Prompt Injections by Design24 Mar
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb
Other references · 14
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
- 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
- 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 161
- 2025Securing AI Agents with Information-Flow Controlcited by 108
- 2024AirGapAgent: Protecting Privacy-Conscious Conversational Agentscited by 90
- 2025Contextual Agent Security: A Policy for Every Purposecited by 55
- 2024CI-Bench: Benchmarking Contextual Integrity of AI Assistants on Synthetic Datacited by 34
- 2025Firewalls to Secure Dynamic LLM Agentic Networkscited by 34
- 2025Contextual Integrity in LLMs via Reasoning and Reinforcement Learningcited by 30
- 2025ConVerse: Benchmarking Contextual Safety in Agent-to-Agent Conversationscited by 14
- 2025Position: Contextual Integrity is Inadequately Applied to Language Modelscited by 11
- 2025Privacy Reasoning in Ambiguous Contextscited by 11
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026Operational Reframing and Approval-Framed Delegation in Multi-Agent LLM Safetycited by 3
- 2026Janus: a Playground for User-Involved Agentic Permission Managementcited by 2
- 2026Red-Teaming the Agentic Red-Teamcited by 0
- 2026AI Snitches Get Glitches: Towards Evading Agentic Surveillancecited by 0
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilitiescited by 0
- 2026Operational Evidence Gaps for LLMs in Fraud Detection and Trust-and-Safety Workflowscited by 0
- 2026Delegation Without Trust: An Empirical Gap Analysis of Identity, Authorization, and Runtime Governance in Multi-Agent LLM Systemscited by 0
- 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systemscited by 0
- 2026Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluationcited by 0
- 2026Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluation21 Sep 2026
- 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems20 Sep 2026
- 2026Delegation Without Trust: An Empirical Gap Analysis of Identity, Authorization, and Runtime Governance in Multi-Agent LLM Systems31 Aug 2026
- 2026Operational Evidence Gaps for LLMs in Fraud Detection and Trust-and-Safety Workflows12 Jul 2026
- 2026Operational Reframing and Approval-Framed Delegation in Multi-Agent LLM Safety8 Jul 2026
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities7 Jul 2026
- 2026Janus: a Playground for User-Involved Agentic Permission Management1 Jul 2026
- 2026AI Snitches Get Glitches: Towards Evading Agentic Surveillance24 Jun 2026
- 2026Red-Teaming the Agentic Red-Team23 Jun 2026