The Unfireable Safety Kernel: Execution-Time AI Alignment for AI Agents and Other Escapable AI Systems
Not on the map. Why?
Abstract (arXiv)
AI agents are granted access to tools, APIs, and other infrastructure, making them active principals in those systems. The dominant approach places controls inside the agent's own runtime: system prompts, output filters, and guardrail libraries. Any control in the agent's address space is reachable by inputs that influence it; this generalizes to any AI system with sufficient reach into its own runtime, a class we term escapable AI systems. We identify four properties that an authorization mechanism must satisfy for architectural control rather than for cooperative requests: process separation, pre-action enforcement on a structurally only path, fail-closed at both the request and system levels, and externalized signed evidence verifiable outside the controlled system's trust boundary. We position this layer as execution-time AI alignment, complementing training-time alignment (RLHF, Constitutional AI) and inference-time alignment. We present the Unfireable Safety Kernel, a Rust reference implementation realizing all four. Its fail-closed invariant is machine-checked at two levels: an SMT theorem (Z3) and an exhaustive bounded-model-checking proof of the production decision function (Kani, 4/4 harnesses). A Python-to-Rust migration was gated on byte-equivalence (1000/1000 fixtures; 17/17 adversarial classes). We evaluate the kernel governing a live, escapable AI system, a deterministic, self-improving world model, against an escape-seeking adversary driving its real self-modification seam: across 1,000 self-modifications, all 704 attempts on the safety-critical core are refused, with no escape; a further 300, under the operator kill switch, are also refused. A separate campaign of 6,240 authorization round-trips had no successful bypass. Against 3 contemporary systems claiming the agent control plane, the agent invokes control; here, it lacks that choice.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 5 of its 10 references that are in the map. First posted on arXiv 24 Jun 2026.
Builds on
5 of 10 references.
References · 5
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
- 2024Alignment faking in large language modelscited by 360
- 2024Frontier Models are Capable of In-context Schemingcited by 287
- 2016The Off-Switch Gamecited by 97
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Sovereign Agentic Loops: Decoupling AI Reasoning from Execution in Real-World Systems
- 2026OpenAgentFlow: Enabling System-Wide Safety Boundaries for Heterogeneous AI Agent Fleets
- 2026Trustworthy Agentic AI Requires Deterministic Architectural Boundaries
- 2026Agent Safety Should Be a Runtime Contract
- 2026Position: AI Safety Requires Effective Controllability
- 2026AgentKernel: The Trust-Native Agentic Operating System
- 2026Structural Enforcement of Goal Integrity in AI Agents via Separation-of-Powers Architecture
- 2026Containment Verification: AI Safety Guarantees Independent of Alignment