Provably Secure Agent Guardrail
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
As large language models transition from bounded generative engines to agents with expansive execution privileges, AI going out of control precipitates a fundamental crisis in artificial intelligence security. Existing defense architectures heavily rely on empirical semantic guardrails and probabilistic large model adjudicators, mechanisms that fail to provide deterministic security lower bounds when facing complex semantic symbol decoupling attacks. To overcome this empirical semantic guardrail dilemma, this paper proposes a new security paradigm for agents based on the fundamental limitations of logical reasoning. Based on this paradigm, we further introduce an executable Proof-Constrained Action (ePCA) framework with a neural symbolic isolation architecture. This framework abandons semantic trust in natural language, forcing agents to losslessly formalize their intentions into first-order logical mathematical constraints before performing physical operations. Empirical evaluations of macroscopic and microscopic two-dimensional dynamic adversarial systems demonstrate that our formal verification mechanism achieves zero attack success rate and zero false positive rate across the evaluated scenarios, with extremely low computational latency. This research provides a conditional formal foundation under explicit system assumptions and an engineering paradigm for constructing the underlying defense foundation for future intelligent systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 34 of its 62 references that are in the map. First posted on arXiv 28 May 2026.
Builds on
34 of 62 references.
Closest ancestors · 8
Same direction, newest first
- 2026Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain9 Apr
- 2026Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClaw13 Mar
- 2026Automating Agent Hijacking via Structural Template Injection18 Feb
- 2025Agentic JWT: A Secure Delegation Protocol for Autonomous AI Agents16 Sep
- 2025AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents9 Sep
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
- 2024AirGapAgent: Protecting Privacy-Conscious Conversational Agents8 May
Other references · 26
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
- 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
- 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
- 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 143
- 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 143
14 more
- 2025One Token to Fool LLM-as-a-Judgecited by 60
- 2023First Three Years of the International Verification of Neural Networks Competition (VNN-COMP)cited by 58
- 2023Can Large Language Models Transform Natural Language Intent into Formal Method Postconditions?cited by 57
- 2023Provably safe systems: the only path to controllable AGIcited by 29
- 2025Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expressioncited by 16
- 2025Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Trainingcited by 12
- 2025VerifyThisBench: Generating Code, Specifications, and Proofs All at Oncecited by 9
- 2025Bridging Natural Language and Formal Specification--Automated Translation of Software Requirements to LTL via Hierarchical Semantics Decomposition Using LLMscited by 9
- 2024Generating API Parameter Security Rules with LLM for API Misuse Detectioncited by 7
- 2025ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusalcited by 6
- 2025INTA: Intent-Based Translation for Network Configuration with LLM Agentscited by 5
- 2026Security in LLM-as-a-Judge: A Comprehensive SoKcited by 5
- 2025Breaking the Prompt Wall (I): A Real-World Case Study of Attacking ChatGPT via Lightweight Prompt Injectioncited by 2
- 2026VeriTrans: Fine-Tuned LLM-Assisted NL-to-PL Translation via a Deterministic Neuro-Symbolic Pipelinecited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Don't Make Models Guess Security and Safety: Symbolic Guardrails for Domain-Specific AI Agents
- 2026PL-Guard: Probabilistic Logic Reasoning for LLM Guardrails
- 2026SingGuard-NSFA: Extensible Guardrails for Agentic AI via Generative Reasoning and Real-Time Classification
- 2025Reasoning Under Threat: Symbolic and Neural Techniques for Cybersecurity Verification
- 2026Extending the Formalism and Theoretical Foundations of Cryptography to AI
- 2026Toward Secure AI-Powered Penetration Testing Agents: Security Threats, Guardrails, and Architectural Perspectives
- 2026Trustworthy Agentic AI: A Comprehensive Cybersecurity and Systems Survey on Threat Landscapes, Defense Architectures, and Open Challenges
- 2026ClawLess: A Security Model of AI Agents