Security Considerations for Artificial Intelligence Agents
Not on the map. Why?
Abstract (arXiv)
This article, a lightly adapted version of Perplexity's response to NIST/CAISI Request for Information 2025-0035, details our observations and recommendations concerning the security of frontier AI agents. These insights are informed by Perplexity's experience operating general-purpose agentic systems used by millions of users and thousands of enterprises in both controlled and open-world environments. Agent architectures change core assumptions around code-data separation, authority boundaries, and execution predictability, creating new confidentiality, integrity, and availability failure modes. We map principal attack surfaces across tools, connectors, hosting boundaries, and multi-agent coordination, with particular emphasis on indirect prompt injection, confused-deputy behavior, and cascading failures in long-running workflows. We then assess current defenses as a layered stack: input-level and model-level mitigations, sandboxed execution, and deterministic policy enforcement for high-consequence actions. Finally, we identify standards and research gaps, including adaptive security benchmarks, policy models for delegation and privilege control, and guidance for secure multi-agent system design aligned with NIST risk management principles.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 12 Mar 2026.
Builds on
26 of 45 references.
References · 26
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
- 2025Defeating Prompt Injections by Designcited by 256
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
- 2024InFoBench: Evaluating Instruction Following Ability in Large Language Modelscited by 128
- 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systemscited by 118
- 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 105
- 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 82
- 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
14 more
- 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchycited by 62
- 2024Get my drift? Catching LLM Task Drift with Activation Deltascited by 61
- 2025DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agentscited by 59
- 2025Contextual Agent Security: A Policy for Every Purposecited by 55
- 2025IHEval: Evaluating Language Models on Following the Instruction Hierarchycited by 48
- 2025IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agentscited by 46
- 2025Control Illusion: The Failure of Instruction Hierarchies in Large Language Modelscited by 31
- 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystemscited by 30
- 2025LLM Agents Should Employ Security Principlescited by 28
- 2024SysBench: Can Large Language Models Follow System Messages?cited by 26
- 2025BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agentscited by 23
- 2024ConfusedPilot: Confused Deputy Risks in RAG-based LLMscited by 22
- 2025ThinkTrap: Denial-of-Service Attacks against Black-box LLM Services via Infinite Thinkingcited by 22
- 2024SoK: Prompt Hacking of Large Language Modelscited by 5
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 68
- 2026Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agentscited by 17
- 2026SUDP: Secret-Use Delegation Protocol for Agentic Systemscited by 2
- 2026Parallax: Why AI Agents That Think Must Never Actcited by 1
- 2026Governing What the EU AI Act Excludes: Accountability for Autonomous AI Agents in Smart City Critical Infrastructurecited by 1
- 2026Constraining Host-Level Abuse in Self-Hosted Computer-Use Agents via TEE-Backed Isolationcited by 1
- 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directionscited by 1
- 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directions14 Jul 2026
- 2026Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agents11 Jul 2026
- 2026Constraining Host-Level Abuse in Self-Hosted Computer-Use Agents via TEE-Backed Isolation7 May 2026
- 2026Governing What the EU AI Act Excludes: Accountability for Autonomous AI Agents in Smart City Critical Infrastructure1 May 2026
- 2026SUDP: Secret-Use Delegation Protocol for Agentic Systems27 Apr 2026
- 2026Parallax: Why AI Agents That Think Must Never Act14 Apr 2026
- 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering9 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2025Systems Security Foundations for Agentic Computing
- 2026Rethinking Agent Security as a Networking Problem
- 2026Securing Agentic AI: From Per-Action Checks to Trajectory Assurance
- 2024Security of AI Agents
- 2026When the Agent Is the Adversary: Architectural Requirements for Agentic AI Containment After the April 2026 Frontier Model Escape
- 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks
- 2026Architecture Matters for Multi-Agent Security
- 2025Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition