ProbeLogits: Kernel-Level LLM Inference Primitives for AI-Native Operating Systems
Not on the map. Why?
Abstract (arXiv)
An OS kernel that runs LLM inference internally can read the model's own next-token logit distribution before any text is generated, and act on it as a governance primitive. I present ProbeLogits, a kernel-level operation that performs a single forward pass and reads specific token logits to classify an agent's action as safe or dangerous, with zero learned parameters. Because the probe reads a logit from the same base model the agent already runs, it removes the second model a fine-tuned guard requires: the marginal cost of a safety check becomes a single logit read. I evaluate ProbeLogits on three base models (Qwen2.5-7B, Llama-3-8B, Mistral-7B) across three external benchmarks (HarmBench, XSTest, ToxicChat). On HarmBench non-copyright, all three reach a 97-99% block rate. On ToxicChat (n=1,000), ProbeLogits attains F1 parity-or-better against Llama Guard 3: Qwen2.5-7B Safe/Dangerous reaches F1=0.812 (+13.7 pp, bootstrap 95% CIs disjoint), Llama-3 matches within CI (+0.4 pp), and Mistral exceeds by +4.4 pp. Classification is a measured 2.4-3.4x faster than Llama Guard 3 (332-556 ms vs. 851-1,142 ms), because it reads a single logit position instead of generating tokens. A calibration strength alpha acts as a deployment-time policy knob rather than a learned hyperparameter, trading recall for precision per operation class. I implement ProbeLogits within Anima OS, a bare-metal x86-64 kernel written in ~285,000 lines of Rust. Because agent actions must pass through kernel-mediated host functions, enforcement operates below the WASM sandbox boundary, making it substantially harder to circumvent than application-layer classifiers.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 13 Apr 2026.
Builds on
20 of 36 references.
References · 20
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
- 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
- 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
8 more
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
- 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 263
- 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 231
- 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 205
- 2023Efficient Guided Generation for Large Language Modelscited by 200
- 2024AIOS: LLM Agent Operating Systemcited by 97
- 2022Petals: Collaborative Inference and Fine-tuning of Large Modelscited by 87
- 2026Right to History: A Sovereignty Kernel for Verifiable AI Agent Executioncited by 7
Built on by
Citing papers per year: 2026: 2.
2 papers.