mlsanityData to 14 Oct 2026

Agent Security is a Systems Problem

arXiv 2605.18991 ↗First posted 18 May 2026cs.CR cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

We take the position that agent security must be approached as a systems problem: the AI model powering the agent must be treated as an untrusted component, and security invariants must be enforced at the system level. Through this lens, efforts to increase model robustness (the dominant viewpoint in the community) are insufficient on their own. Instead, we must complement existing efforts with techniques from the systems security domain. Based on our experience as cybersecurity researchers in operating systems, networks, formal methods, and adversarial machine learning, we articulate a set of core principles, grounded in decades of systems security research, that provide a foundation for designing agentic systems with predictable guarantees. As evidence, we analyze eleven representative real-world attacks on agents and discuss how systems principles, if realized, could have prevented these attacks. We also identify the research challenges that stand in the way of implementing these principles in agents.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 31 of its 70 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

31 of 70 references.

References · 31

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  3. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  4. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  5. 2021Causal Abstractions of Neural Networkscited by 422
  6. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
  7. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  8. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 210
  9. 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoningcited by 151
  10. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 143
  11. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 129
  12. 2025Securing AI Agents with Information-Flow Controlcited by 121
19 more
  1. 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 81
  2. 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchycited by 67
  3. 2025ACE: A Security Architecture for LLM-Integrated App Systemscited by 55
  4. 2024PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Railscited by 52
  5. 2025VeriGuard: Enhancing LLM Agent Safety via Verified Code Generationcited by 49
  6. 2025SAGA: A Security Architecture for Governing AI Agentic Systemscited by 45
  7. 2025AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agentscited by 42
  8. 2025A Critical Evaluation of Defenses against Prompt Injection Attackscited by 36
  9. 2024Cedar: A New Language for Expressive, Fast, Safe, and Analyzable Authorization (Extended Version)cited by 34
  10. 2026Formal Policy Enforcement for Real-World Agentic Systemscited by 33
  11. 2026MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attackscited by 21
  12. 2025ceLLMate: Sandboxing Browser AI Agentscited by 16
  13. 2025May I have your Attention? Breaking Fine-Tuning based Prompt Injection Defenses using Architecture-Aware Attackscited by 15
  14. 2025Manipulating LLM Web Agents with Indirect Prompt Injection Attack via HTML Accessibility Treecited by 14
  15. 2025How Not to Detect Prompt Injections with an LLMcited by 12
  16. 2023Information Flow Control in Machine Learning through Modular Model Architecturecited by 11
  17. 2026Optimizing Agent Planning for Security and Autonomycited by 10
  18. 2024Validating Mechanistic Interpretations: An Axiomatic Approachcited by 4
  19. 2026Trojan-Speak: Bypassing Constitutional Classifiers with No Jailbreak Tax via Adversarial Finetuningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Systems Security Foundations for Agentic Computing
  2. 2026Rethinking Agent Security as a Networking Problem
  3. 2025A Survey on Agentic Security: Applications, Threats and Defenses
  4. 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
  5. 2026Securing Agentic AI: From Per-Action Checks to Trajectory Assurance
  6. 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks
  7. 2025Agentic AI for Cyber Resilience: A New Security Paradigm and Its System-Theoretic Foundations
  8. 2026To Defend Against Cyber Attacks, We Must Teach AI Agents to Hack