Systems Security Foundations for Agentic Computing
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
In recent years, agentic artificial intelligence (AI) systems are becoming increasingly widespread. These systems allow agents to use various tools, such as web browsers, compilers, and more. However, despite their popularity, agentic AI systems also introduce a myriad of security concerns, due to their constant interaction with third-party servers. For example, a malicious adversary can cause data exfiltration by executing prompt injection attacks, as well as other unwarranted behavior. These security concerns have recently motivated researchers to improve the safety and reliability of agentic systems. However, most of the literature on this topic is from the AI standpoint and lacks the system-security perspective and guarantees. In this work, we begin bridging this gap and present an analysis through the lens of classic cybersecurity research. Specifically, motivated by decades of progress in this domain, we identify short- and long-term research problems in agentic AI safety by examining end-to-end security properties of entire systems, rather than standalone AI models running in isolation. Our key goal is to examine where research challenges arise when applying traditional security principles in the context of AI agents and, as a secondary goal, distill these ideas for AI practitioners. Furthermore, we extensively cover 11 case studies of real-world attacks on agentic systems, as well as define a series of new research problems that are specific to this important domain.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 62 of its 118 references that are in the map. First posted on arXiv 1 Dec 2025.
Builds on
62 of 118 references.
Closest ancestors · 18
Same direction, newest first
- 2025VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation3 Oct
- 2025DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents13 Jun
- 2025Design Patterns for Securing LLM Agents against Prompt Injections10 Jun
- 2025LLM Agents Should Employ Security Principles29 May
- 2025SAGA: A Security Architecture for Governing AI Agentic Systems27 Apr
- 2025DoomArena: A framework for Testing AI Agents Against Evolving Security Threats18 Apr
- 2025Progent: Securing AI Agents with Privilege Control16 Apr
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
- 2025Defeating Prompt Injections by Design24 Mar
- 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage13 Feb
- 2024Attacking Vision-Language Computer Agents via Pop-ups4 Nov
- 2024AdvAgent: Controllable Blackbox Red-teaming on Web Agents22 Oct
6 more
- 2024Imprompter: Tricking LLM Agents into Improper Tool Use19 Oct
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage17 Sep
- 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification30 Jul
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
Other references · 44
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
- 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
- 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 439
- 2017Safe Reinforcement Learning via Shieldingcited by 427
32 more
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
- 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 307
- 2018Certified Defenses against Adversarial Examplescited by 305
- 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
- 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 254
- 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
- 2021Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Robustness Verificationcited by 224
- 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
- 2018Semidefinite relaxations for certifying robustness to adversarial examplescited by 178
- 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
- 2020Neural Lyapunov Controlcited by 142
- 2019Branch and Bound for Piecewise Linear Neural Network Verificationcited by 138
- 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
- 2025Securing AI Agents with Information-Flow Controlcited by 108
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
- 2024System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspectivecited by 91
- 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 82
- 2017Verifying Properties of Binarized Deep Neural Networkscited by 77
- 2021Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificatescited by 71
- 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
- 2021Introduction to Neural Network Verificationcited by 63
- 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchycited by 62
- 2025Contextual Agent Security: A Policy for Every Purposecited by 55
- 2025ACE: A Security Architecture for LLM-Integrated App Systemscited by 52
- 2024LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Modelscited by 49
- 2025Demonstrating specification gaming in reasoning modelscited by 48
- 2024Cedar: A New Language for Expressive, Fast, Safe, and Analyzable Authorization (Extended Version)cited by 31
- 2025Defending Against Prompt Injection With a Few DefensiveTokenscited by 26
- 2025May I have your Attention? Breaking Fine-Tuning based Prompt Injection Defenses using Architecture-Aware Attackscited by 14
- 2023Information Flow Control in Machine Learning through Modular Model Architecturecited by 11
- 2025How Not to Detect Prompt Injections with an LLMcited by 11
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attackscited by 9
- 2026Type-Checked Compliance: Deterministic Guardrails for Agentic Financial Systems Using Lean 4 Theorem Provingcited by 2
- 2026LACUNA: Safe Agents as Recursive Program Holescited by 1
- 2026The Containment Gap: How Deployed Agentic AI Frameworks Fail Public-Facing Safety Requirementscited by 1
- 2026LLMs + Security = Troublecited by 0
- 2026Lessons from Penetration Tests on Large-Scale Agent Systemscited by 0
- 2026GIF: Locally Sound Geometric Information Flow Control for LLMscited by 0
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilitiescited by 0
- 2026Reflections on Trusting Trust, Revisited: Contaminating Self-Modifying AI Coding Agents with Poisoned Benchmarkscited by 0
- 2026Reflections on Trusting Trust, Revisited: Contaminating Self-Modifying AI Coding Agents with Poisoned Benchmarks15 Sep 2026
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities7 Jul 2026
- 2026GIF: Locally Sound Geometric Information Flow Control for LLMs22 Jun 2026
- 2026The Containment Gap: How Deployed Agentic AI Frameworks Fail Public-Facing Safety Requirements11 Jun 2026
- 2026LACUNA: Safe Agents as Recursive Program Holes27 May 2026
- 2026Lessons from Penetration Tests on Large-Scale Agent Systems26 May 2026
- 2026Type-Checked Compliance: Deterministic Guardrails for Agentic Financial Systems Using Lean 4 Theorem Proving1 Apr 2026
- 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks31 Mar 2026
- 2026LLMs + Security = Trouble9 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
- 2026Agent Security is a Systems Problem
- 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
- 2025A Safety and Security Framework for Real-World Agentic Systems
- 2025A Survey on Agentic Security: Applications, Threats and Defenses
- 2025Agentic AI for Cyber Resilience: A New Security Paradigm and Its System-Theoretic Foundations
- 2026A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypes
- 2026Rethinking Agent Security as a Networking Problem