Formal Policy Enforcement for Real-World Agentic Systems
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Security policy enforcement in contemporary agentic systems predominantly consists of embedding natural-language policies within an agent's system prompt and delegating compliance to the agent's reasoning. This approach admits no formal enforcement guarantee and cannot express policies whose satisfaction depends on the causal history of an execution, a gap that becomes acute in multi-agent systems, where enforcement must reason across agents. We argue that policy enforcement in agentic systems is most naturally understood as a cross-cutting concern, and propose a framework grounded in aspect-oriented programming that specifies policies independent of the agent's reasoning and enforces them at every policy-relevant decision. Policies are written in Datalog over a set of abstract predicates describing the execution context, an observability service governed by a formal assume/guarantee contract maintains these predicates, and a reference monitor consults the policy at each action to produce an enforcement decision. When the environment contract holds, enforcement decisions coincide with the policy's intended semantics. We adopt Datalog as the policy language, a natural fit because it supports declarative rule specification, admits recursion for policies over transitive relationships, and yields deterministic enforcement. Datalog further admits tractable static analyses for contradiction, redundancy, subsumption, and conditional reachability, enabling authors to verify policy intent and surface ambiguities inherent in natural-language specifications. We realize the framework in FORGE, which enforces policies over agentic deployments without modification to the underlying agents. We evaluate FORGE on three case studies: information flow policies for prompt injection defense, approval workflows in a multi-agent pharmacovigilance system, and organizational policies for customer service.
Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 24 of its 61 references that are in the map. First posted on arXiv 18 Feb 2026.
Builds on
24 of 61 references.
Closest ancestors · 4
Same direction, newest first
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
- 2025Defeating Prompt Injections by Design24 Mar
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
Other references · 20
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
- 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
- 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
- 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
- 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
8 more
- 2025Securing AI Agents with Information-Flow Controlcited by 108
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
- 2024Cedar: A New Language for Expressive, Fast, Safe, and Analyzable Authorization (Extended Version)cited by 31
- 2025A Critical Evaluation of Defenses against Prompt Injection Attackscited by 31
- 2017Practical Whole-System Provenance Capturecited by 19
- 2025Policy-as-Prompt: Turning AI Governance Rules into Guardrails for AI Agentscited by 12
- 2024MALADE: Orchestration of LLM-powered Agents with Retrieval Augmented Generation for Pharmacovigilancecited by 11
- 2025How Not to Detect Prompt Injections with an LLMcited by 11
Built on by
Citing papers per year: 2026: 31.
31 papers.
Sort:
By area:
- 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecyclecited by 21
- 2026Before the Tool Call: Deterministic Pre-Action Authorization for Autonomous AI Agentscited by 12
- 2026Reason Less, Verify More: Deterministic Gates Recover a Silent Policy-Violation Failure Mode in Tool-Using LLM Agentscited by 8
- 2026Context-to-Execution Integrity for LLM Agentscited by 7
- 2026Lingering Authority: Revocable Resource-and-Effect Capabilities for Coding Agentscited by 5
- 2026CONTINUITY: Security-Context Contracts for Composable LLM Agent Controlscited by 3
- 2026Agent Security is a Systems Problemcited by 2
- 2026SecureClaw: Clawing Back Control of LLM Agentscited by 2
- 2026Janus: a Playground for User-Involved Agentic Permission Managementcited by 2
- 2026How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcementcited by 2
- 2026CatchBench: When Can an Agent Failure Be Caught?cited by 2
- 2026AgentFlow: A Flow-Centric Policy Language and Framework for Securing LLM Agent Systemscited by 2
19 more
- 2026Causality Laundering: Denial-Feedback Leakage in Tool-Calling LLM Agentscited by 1
- 2026PolicyBank: Evolving Policy Understanding for LLM Agentscited by 1
- 2026Owner-Harm: A Missing Threat Model for AI Agent Safetycited by 1
- 2026Constraining Host-Level Abuse in Self-Hosted Computer-Use Agents via TEE-Backed Isolationcited by 1
- 2026Prompts Don't Protect: Architectural Enforcement via MCP Proxy for LLM Tool Access Controlcited by 1
- 2026Efficient and Sound Probabilistic Verification for AI Agentscited by 1
- 2026Proof-of-Execution Memory: Defending LLM Agents Against Forged-Reasoning Attacks by Verifying What Actually Happenedcited by 1
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilitiescited by 0
- 2026Verification of Adaptive Agentic Controllers through Finite Rule Revisioncited by 0
- 2026Delegation Without Trust: An Empirical Gap Analysis of Identity, Authorization, and Runtime Governance in Multi-Agent LLM Systemscited by 0
- 2026Where Reliability Lives: Experimental Localisation of Behavioural Properties in an Agent Systemcited by 0
- 2026Versioned Transitive Dependency-Closure Binding and Operation-Time Effect Governance for Agent Skills: ClosureBoundcited by 0
- 2026How Strongly Should Task State Influence an LLM Agent?cited by 0
- 2026Authorization Closure Graph: Minimal Repair for LLM Agents with Evolving User Instructionscited by 0
- 2026Certified Multi-Source Integrity for Structured Agent Actionscited by 0
- 2026Audience-Bound Persistent Memory: Authorization Across the Memory Lifecyclecited by 0
- 2026Sapien: A Stateful Policy Engine for Autonomous AI Agentscited by 0
- 2026PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agentscited by 0
- 2026NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agentscited by 0
- 2026NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents8 Oct 2026
- 2026PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents1 Oct 2026
- 2026Sapien: A Stateful Policy Engine for Autonomous AI Agents30 Sep 2026
- 2026Audience-Bound Persistent Memory: Authorization Across the Memory Lifecycle28 Sep 2026
- 2026Certified Multi-Source Integrity for Structured Agent Actions28 Sep 2026
- 2026Authorization Closure Graph: Minimal Repair for LLM Agents with Evolving User Instructions26 Sep 2026
- 2026How Strongly Should Task State Influence an LLM Agent?22 Sep 2026
- 2026Versioned Transitive Dependency-Closure Binding and Operation-Time Effect Governance for Agent Skills: ClosureBound5 Sep 2026
- 2026CONTINUITY: Security-Context Contracts for Composable LLM Agent Controls4 Sep 2026
- 2026Where Reliability Lives: Experimental Localisation of Behavioural Properties in an Agent System2 Sep 2026
- 2026Delegation Without Trust: An Empirical Gap Analysis of Identity, Authorization, and Runtime Governance in Multi-Agent LLM Systems31 Aug 2026
- 2026AgentFlow: A Flow-Centric Policy Language and Framework for Securing LLM Agent Systems24 Aug 2026
19 more
- 2026CatchBench: When Can an Agent Failure Be Caught?24 Aug 2026
- 2026Proof-of-Execution Memory: Defending LLM Agents Against Forged-Reasoning Attacks by Verifying What Actually Happened17 Aug 2026
- 2026How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcement15 Jul 2026
- 2026Reason Less, Verify More: Deterministic Gates Recover a Silent Policy-Violation Failure Mode in Tool-Using LLM Agents8 Jul 2026
- 2026Verification of Adaptive Agentic Controllers through Finite Rule Revision7 Jul 2026
- 2026Context-to-Execution Integrity for LLM Agents7 Jul 2026
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities7 Jul 2026
- 2026Janus: a Playground for User-Involved Agentic Permission Management1 Jul 2026
- 2026Lingering Authority: Revocable Resource-and-Effect Capabilities for Coding Agents21 Jun 2026
- 2026Efficient and Sound Probabilistic Verification for AI Agents18 Jun 2026
- 2026SecureClaw: Clawing Back Control of LLM Agents8 Jun 2026
- 2026Agent Security is a Systems Problem18 May 2026
- 2026Prompts Don't Protect: Architectural Enforcement via MCP Proxy for LLM Tool Access Control18 May 2026
- 2026Constraining Host-Level Abuse in Self-Hosted Computer-Use Agents via TEE-Backed Isolation7 May 2026
- 2026Owner-Harm: A Missing Threat Model for AI Agent Safety20 Apr 2026
- 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle17 Apr 2026
- 2026PolicyBank: Evolving Policy Understanding for LLM Agents16 Apr 2026
- 2026Causality Laundering: Denial-Feedback Leakage in Tool-Calling LLM Agents5 Apr 2026
- 2026Before the Tool Call: Deterministic Pre-Action Authorization for Autonomous AI Agents21 Mar 2026
- 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecyclecited by 21
- 2026Context-to-Execution Integrity for LLM Agentscited by 7
- 2026Lingering Authority: Revocable Resource-and-Effect Capabilities for Coding Agentscited by 5
- 2026CONTINUITY: Security-Context Contracts for Composable LLM Agent Controlscited by 3
- 2026SecureClaw: Clawing Back Control of LLM Agentscited by 2
- 2026Janus: a Playground for User-Involved Agentic Permission Managementcited by 2
- 2026How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcementcited by 2
- 2026AgentFlow: A Flow-Centric Policy Language and Framework for Securing LLM Agent Systemscited by 2
- 2026Causality Laundering: Denial-Feedback Leakage in Tool-Calling LLM Agentscited by 1
- 2026PolicyBank: Evolving Policy Understanding for LLM Agentscited by 1
- 2026Owner-Harm: A Missing Threat Model for AI Agent Safetycited by 1
- 2026Constraining Host-Level Abuse in Self-Hosted Computer-Use Agents via TEE-Backed Isolationcited by 1
12 most cited of 21 All 21 (JSON)
- 2026PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents1 Oct 2026
- 2026Certified Multi-Source Integrity for Structured Agent Actions28 Sep 2026
- 2026Authorization Closure Graph: Minimal Repair for LLM Agents with Evolving User Instructions26 Sep 2026
- 2026How Strongly Should Task State Influence an LLM Agent?22 Sep 2026
- 2026CONTINUITY: Security-Context Contracts for Composable LLM Agent Controls4 Sep 2026
- 2026Where Reliability Lives: Experimental Localisation of Behavioural Properties in an Agent System2 Sep 2026
- 2026Delegation Without Trust: An Empirical Gap Analysis of Identity, Authorization, and Runtime Governance in Multi-Agent LLM Systems31 Aug 2026
- 2026AgentFlow: A Flow-Centric Policy Language and Framework for Securing LLM Agent Systems24 Aug 2026
- 2026Proof-of-Execution Memory: Defending LLM Agents Against Forged-Reasoning Attacks by Verifying What Actually Happened17 Aug 2026
- 2026How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcement15 Jul 2026
- 2026Verification of Adaptive Agentic Controllers through Finite Rule Revision7 Jul 2026
- 2026Context-to-Execution Integrity for LLM Agents7 Jul 2026
12 newest of 21 All 21 (JSON)
- 2026Before the Tool Call: Deterministic Pre-Action Authorization for Autonomous AI Agentscited by 12
- 2026Agent Security is a Systems Problemcited by 2
- 2026CatchBench: When Can an Agent Failure Be Caught?cited by 2
- 2026Prompts Don't Protect: Architectural Enforcement via MCP Proxy for LLM Tool Access Controlcited by 1
- 2026Efficient and Sound Probabilistic Verification for AI Agentscited by 1
- 2026Versioned Transitive Dependency-Closure Binding and Operation-Time Effect Governance for Agent Skills: ClosureBoundcited by 0
- 2026Audience-Bound Persistent Memory: Authorization Across the Memory Lifecyclecited by 0
- 2026Sapien: A Stateful Policy Engine for Autonomous AI Agentscited by 0
- 2026NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agentscited by 0
- 2026NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents8 Oct 2026
- 2026Sapien: A Stateful Policy Engine for Autonomous AI Agents30 Sep 2026
- 2026Audience-Bound Persistent Memory: Authorization Across the Memory Lifecycle28 Sep 2026
- 2026Versioned Transitive Dependency-Closure Binding and Operation-Time Effect Governance for Agent Skills: ClosureBound5 Sep 2026
- 2026CatchBench: When Can an Agent Failure Be Caught?24 Aug 2026
- 2026Efficient and Sound Probabilistic Verification for AI Agents18 Jun 2026
- 2026Agent Security is a Systems Problem18 May 2026
- 2026Prompts Don't Protect: Architectural Enforcement via MCP Proxy for LLM Tool Access Control18 May 2026
- 2026Before the Tool Call: Deterministic Pre-Action Authorization for Autonomous AI Agents21 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026VIGIL: Runtime Enforcement of Behavioral Specifications in AI Agent Skills
- 2025QuadSentinel: Sequent Safety for Machine-Checkable Control in Multi-agent Systems
- 2026Autoformalization of Agent Instructions into Policy-as-Code
- 2026Alignment Contracts for Agentic Security Systems
- 2026Runtime Governance for AI Agents: Policies on Paths
- 2026ETAS: An Effect-Typed Language for Agent Systems
- 2025ARPaCCino: An Agentic-RAG for Policy as Code Compliance
- 2026STAGE: Stateful Translation to Agentic Graph Execution with Policy-Scoped Context and Deterministic Control