When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Large language model agents are now privileged principals that take consequential actions: editing code repositories, operating inboxes, completing purchases. Their authority is kernel-grade, but it comes without what classical systems security requires: a trusted mediator interposed on every access. Operating-system vendors are now rebuilding the platform around this de-facto agent kernel, inheriting complete mediation as a design problem. We systematize the security of such systems around a single distinction: a crossing mediated over provenance admits a deterministic check, while one over content semantics does not. A trust-boundary taxonomy locates where mediation must occur and isolates the central mediation gap at two kinds of semantic judgment: distinguishing data from instruction in untrusted input, and an authorized action from an unauthorized one. We argue that this gap leaves an irreducible residual of undetected attacks wherever inputs and actions are not restricted in advance to an enumerated set. The same distinction makes attack-success statistics actionable, placing each number on a spectrum from deployment debt (a sound deterministic mediator left unused) to a structural gap (no such mediator known). We systematize defenses across runtime monitoring, architectural separation, and authorization, and show that current evaluations tend to overstate deployed security through evaluation-validity failures. Finally, we carry that analysis forward beyond the de-facto kernel, to an architecture in which the model itself becomes the arbitration core, and derive the design constraints, open challenges, and research agenda for a security-first AI-native OS.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 143 of its 194 references that are in the map. First posted on arXiv 20 Sep 2026.
Builds on
143 of 194 references.
Closest ancestors · 65
Same direction, newest first
- 2026Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens29 Jun
- 2026Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAG20 Jun
- 2026AgenticOS: An Intent-Oriented Secure Operating System Architecture for Autonomous AI Agents19 Jun
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation9 Jun
- 2026SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces31 May
- 2026Reframing LLM Agent Security as an Agent-Human Interaction Problem23 May
- 2026ASEval: Automated Trajectory-Level Security Testing for Autonomous Agents21 May
- 2026ADR: An Agentic Detection System for Enterprise Agentic AI Security17 May
- 2026SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors15 May
- 2026Toward Securing AI Agents Like Operating Systems14 May
- 2026MemLineage: Lineage-Guided Enforcement for LLM Agent Memory14 May
- 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework25 Apr
38 more
- 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection13 Apr
- 2026Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain9 Apr
- 2026Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems3 Apr
- 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomy24 Mar
- 2026How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition16 Mar
- 2026From Storage to Steering: Memory Control Flow Attacks on LLM Agents16 Mar
- 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey11 Mar
- 2026ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correction24 Feb
- 2026Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections17 Feb
- 2026NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agents21 Jan
- 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework17 Jan
- 2026Beyond Max Tokens: Stealthy Resource Amplification via Tool Calling Chains in LLM Agents16 Jan
- 2025MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval18 Dec
- 2025SoK: Trust-Authorization Mismatch in LLM Agent Interactions7 Dec
- 2025A Vision for Access Control in LLM-based Agent Systems13 Oct
- 2025Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols10 Oct
- 2025AI Agents with Decentralized Identifiers and Verifiable Credentials1 Oct
- 2025A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory29 Sep
- 2025MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers19 Aug
- 2025AgentSight: System-Level Observability for AI Agents Using eBPF2 Aug
- 2025Manipulating LLM Web Agents with Indirect Prompt Injection Attack via HTML Accessibility Tree20 Jul
- 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise9 Jul
- 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety8 Jul
- 2025SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents17 Jun
- 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents17 Jun
- 2025Design Patterns for Securing LLM Agents against Prompt Injections10 Jun
- 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents3 Jun
- 2025Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystem31 May
- 2025RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments28 May
- 2025A Novel Zero-Trust Identity Framework for Agentic AI: Decentralized Authentication and Fine-Grained Access Control25 May
- 2025WebInject: Prompt Injection Attack to Web Agents16 May
- 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
- 2025Prompt Injection Attack to Tool Selection in LLM Agents28 Apr
- 2025SAGA: A Security Architecture for Governing AI Agentic Systems27 Apr
- 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
- 2025Progent: Securing AI Agents with Privilege Control16 Apr
- 2025Defeating Prompt Injections by Design24 Mar
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar
50 newest of 65 All 65 (JSON)
Other references · 78
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
- 2024Scaling and evaluating sparse autoencoderscited by 530
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 338
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
- 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
- 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
- 2024Frontier Models are Capable of In-context Schemingcited by 252
38 more
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
- 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 222
- 2023AI Control: Improving Safety Despite Intentional Subversioncited by 207
- 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
- 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 172
- 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
- 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
- 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
- 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
- 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 117
- 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 113
- 2025Securing AI Agents with Information-Flow Controlcited by 108
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
- 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
- 2024AIOS: LLM Agent Operating Systemcited by 94
- 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Modelscited by 84
- 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 82
- 2024Visibility into AI Agentscited by 77
- 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
- 2024On the Role of Attention Heads in Large Language Model Safetycited by 70
- 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 59
- 2024Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applicationscited by 58
- 2025Contextual Agent Security: A Policy for Every Purposecited by 55
- 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 49
- 2025Monitoring Monitorabilitycited by 47
- 2026AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?cited by 45
- 2024Obfuscated Activations Bypass LLM Latent-Space Defensescited by 40
- 2025Infrastructure for AI Agentscited by 40
- 2025Lessons from Defending Gemini Against Indirect Prompt Injectionscited by 40
- 2025The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agentscited by 39
- 2026Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreakscited by 36
- 2023LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystemcited by 35
- 2025Practical Poisoning Attacks against Retrieval-Augmented Generationcited by 35
- 2024Denial-of-Service Poisoning Attacks against Large Language Modelscited by 34
- 2025EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM Systemcited by 26
- 2024GoEX: Perspectives and Designs Towards a Runtime for Autonomous LLM Applicationscited by 22
- 2026Reasoning Models Struggle to Control their Chains of Thoughtcited by 22
- 2025ASIDE: Architectural Separation of Instructions and Data in Language Modelscited by 21
50 most cited of 78 All 78 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AgentKernel: The Trust-Native Agentic Operating System
- 2026Trustworthy Agentic AI Requires Deterministic Architectural Boundaries
- 2026Securing LLM Agents Need Intent-to-Execution Integrity
- 2026From Craft to Kernel: A Governance-First Execution Architecture and Semantic ISA for Agentic Computers
- 2026ElephantAgent: Contextual State Continuity in Agentic Systems
- 2026Agent-Fence: Mapping Security Vulnerabilities Across Deep Research Agents
- 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents
- 2026Recognition Without Enforcement: Configuration-Dependent Failures in LLM Agent Instruction Arbitration and External Control