mlsanityData to 7 Oct 2026

When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems

arXiv 2609.23700 ↗First posted 20 Sep 2026cs.CR cs.AI cs.OScited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Large language model agents are now privileged principals that take consequential actions: editing code repositories, operating inboxes, completing purchases. Their authority is kernel-grade, but it comes without what classical systems security requires: a trusted mediator interposed on every access. Operating-system vendors are now rebuilding the platform around this de-facto agent kernel, inheriting complete mediation as a design problem. We systematize the security of such systems around a single distinction: a crossing mediated over provenance admits a deterministic check, while one over content semantics does not. A trust-boundary taxonomy locates where mediation must occur and isolates the central mediation gap at two kinds of semantic judgment: distinguishing data from instruction in untrusted input, and an authorized action from an unauthorized one. We argue that this gap leaves an irreducible residual of undetected attacks wherever inputs and actions are not restricted in advance to an enumerated set. The same distinction makes attack-success statistics actionable, placing each number on a spectrum from deployment debt (a sound deterministic mediator left unused) to a structural gap (no such mediator known). We systematize defenses across runtime monitoring, architectural separation, and authorization, and show that current evaluations tend to overstate deployed security through evaluation-validity failures. Finally, we carry that analysis forward beyond the de-facto kernel, to an architecture in which the model itself becomes the arbitration core, and derive the design constraints, open challenges, and research agenda for a security-first AI-native OS.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 143 of its 194 references that are in the map. First posted on arXiv 20 Sep 2026.

Builds on

143 of 194 references.

Closest ancestors · 65

Same direction, newest first

  1. 2026Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens29 Jun
  2. 2026Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAG20 Jun
  3. 2026AgenticOS: An Intent-Oriented Secure Operating System Architecture for Autonomous AI Agents19 Jun
  4. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation9 Jun
  5. 2026SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces31 May
  6. 2026Reframing LLM Agent Security as an Agent-Human Interaction Problem23 May
  7. 2026ASEval: Automated Trajectory-Level Security Testing for Autonomous Agents21 May
  8. 2026ADR: An Agentic Detection System for Enterprise Agentic AI Security17 May
  9. 2026SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors15 May
  10. 2026Toward Securing AI Agents Like Operating Systems14 May
  11. 2026MemLineage: Lineage-Guided Enforcement for LLM Agent Memory14 May
  12. 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework25 Apr
38 more
  1. 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection13 Apr
  2. 2026Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain9 Apr
  3. 2026Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems3 Apr
  4. 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomy24 Mar
  5. 2026How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition16 Mar
  6. 2026From Storage to Steering: Memory Control Flow Attacks on LLM Agents16 Mar
  7. 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey11 Mar
  8. 2026ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correction24 Feb
  9. 2026Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections17 Feb
  10. 2026NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agents21 Jan
  11. 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework17 Jan
  12. 2026Beyond Max Tokens: Stealthy Resource Amplification via Tool Calling Chains in LLM Agents16 Jan
  13. 2025MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval18 Dec
  14. 2025SoK: Trust-Authorization Mismatch in LLM Agent Interactions7 Dec
  15. 2025A Vision for Access Control in LLM-based Agent Systems13 Oct
  16. 2025Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols10 Oct
  17. 2025AI Agents with Decentralized Identifiers and Verifiable Credentials1 Oct
  18. 2025A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory29 Sep
  19. 2025MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers19 Aug
  20. 2025AgentSight: System-Level Observability for AI Agents Using eBPF2 Aug
  21. 2025Manipulating LLM Web Agents with Indirect Prompt Injection Attack via HTML Accessibility Tree20 Jul
  22. 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise9 Jul
  23. 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety8 Jul
  24. 2025SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents17 Jun
  25. 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents17 Jun
  26. 2025Design Patterns for Securing LLM Agents against Prompt Injections10 Jun
  27. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents3 Jun
  28. 2025Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystem31 May
  29. 2025RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments28 May
  30. 2025A Novel Zero-Trust Identity Framework for Agentic AI: Decentralized Authentication and Fine-Grained Access Control25 May
  31. 2025WebInject: Prompt Injection Attack to Web Agents16 May
  32. 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
  33. 2025Prompt Injection Attack to Tool Selection in LLM Agents28 Apr
  34. 2025SAGA: A Security Architecture for Governing AI Agentic Systems27 Apr
  35. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
  36. 2025Progent: Securing AI Agents with Privilege Control16 Apr
  37. 2025Defeating Prompt Injections by Design24 Mar
  38. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar

50 newest of 65 All 65 (JSON)

Other references · 78

  1. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  2. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  3. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  4. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  5. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  6. 2024Scaling and evaluating sparse autoencoderscited by 530
  7. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  8. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 338
  9. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  10. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  11. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
  12. 2024Frontier Models are Capable of In-context Schemingcited by 252
38 more
  1. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
  2. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 222
  3. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 207
  4. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
  5. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 172
  6. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  7. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  8. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
  9. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
  10. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 117
  11. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 113
  12. 2025Securing AI Agents with Information-Flow Controlcited by 108
  13. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
  14. 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
  15. 2024AIOS: LLM Agent Operating Systemcited by 94
  16. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Modelscited by 84
  17. 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 82
  18. 2024Visibility into AI Agentscited by 77
  19. 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
  20. 2024On the Role of Attention Heads in Large Language Model Safetycited by 70
  21. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 59
  22. 2024Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applicationscited by 58
  23. 2025Contextual Agent Security: A Policy for Every Purposecited by 55
  24. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 49
  25. 2025Monitoring Monitorabilitycited by 47
  26. 2026AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?cited by 45
  27. 2024Obfuscated Activations Bypass LLM Latent-Space Defensescited by 40
  28. 2025Infrastructure for AI Agentscited by 40
  29. 2025Lessons from Defending Gemini Against Indirect Prompt Injectionscited by 40
  30. 2025The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agentscited by 39
  31. 2026Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreakscited by 36
  32. 2023LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystemcited by 35
  33. 2025Practical Poisoning Attacks against Retrieval-Augmented Generationcited by 35
  34. 2024Denial-of-Service Poisoning Attacks against Large Language Modelscited by 34
  35. 2025EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM Systemcited by 26
  36. 2024GoEX: Perspectives and Designs Towards a Runtime for Autonomous LLM Applicationscited by 22
  37. 2026Reasoning Models Struggle to Control their Chains of Thoughtcited by 22
  38. 2025ASIDE: Architectural Separation of Instructions and Data in Language Modelscited by 21

50 most cited of 78 All 78 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentKernel: The Trust-Native Agentic Operating System
  2. 2026Trustworthy Agentic AI Requires Deterministic Architectural Boundaries
  3. 2026Securing LLM Agents Need Intent-to-Execution Integrity
  4. 2026From Craft to Kernel: A Governance-First Execution Architecture and Semantic ISA for Agentic Computers
  5. 2026ElephantAgent: Contextual State Continuity in Agentic Systems
  6. 2026Agent-Fence: Mapping Security Vulnerabilities Across Deep Research Agents
  7. 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents
  8. 2026Recognition Without Enforcement: Configuration-Dependent Failures in LLM Agent Instruction Arbitration and External Control