mlsanityData to 7 Oct 2026

Trustworthy Agentic AI Requires Deterministic Architectural Boundaries

arXiv 2602.09947 ↗First posted 10 Feb 2026cs.CRcited by 3

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Current agentic AI architectures are fundamentally incompatible with the security and epistemological requirements of high-stakes scientific workflows. The problem is not inadequate alignment or insufficient guardrails, it is architectural: autoregressive language models process all tokens uniformly, making deterministic command--data separation unattainable through training alone. We argue that deterministic, architectural enforcement, not probabilistic learned behavior, is a necessary condition for trustworthy AI-assisted science. We introduce the Trinity Defense Architecture, which enforces security through three mechanisms: action governance via a finite action calculus with reference-monitor enforcement, information-flow control via mandatory access labels preventing cross-scope leakage, and privilege separation isolating perception from execution. We show that without unforgeable provenance and deterministic mediation, the ``Lethal Trifecta'' (untrusted inputs, privileged data access, external action capability) turns authorization security into an exploit-discovery problem: training-based defenses may reduce empirical attack rates but cannot provide deterministic guarantees. The ML community must recognize that alignment is insufficient for authorization security, and that architectural mediation is required before agentic AI can be safely deployed in consequential scientific domains.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 25 of its 58 references that are in the map. First posted on arXiv 10 Feb 2026.

Builds on

25 of 58 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  2. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  10. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  11. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  12. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
11 more
  1. 2022Red Teaming Language Models with Language Modelscited by 983
  2. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  3. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  4. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  5. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
  6. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 260
  7. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
  8. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  9. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
  10. 2023Evaluating Language-Model Agents on Realistic Autonomous Taskscited by 94
  11. 2024Cedar: A New Language for Expressive, Fast, Safe, and Analyzable Authorization (Extended Version)cited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Trustworthy Agentic AI: A Comprehensive Cybersecurity and Systems Survey on Threat Landscapes, Defense Architectures, and Open Challenges
  2. 2026AgentKernel: The Trust-Native Agentic Operating System
  3. 2026Verifiable Agentic Infrastructure: Proof-Derived Authorization for Sovereign AI Systems
  4. 2026A Policy Algebra for Trust-Preserving Agentic AI Execution
  5. 2026Authenticated Workflows: A Systems Approach to Protecting Agentic AI
  6. 2026From Craft to Kernel: A Governance-First Execution Architecture and Semantic ISA for Agentic Computers
  7. 2025Safe, Untrusted, "Proof-Carrying" AI Agents: toward the agentic lakehouse
  8. 2026Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security