Policy-as-Prompt: Turning AI Governance Rules into Guardrails for AI Agents
On the map: LLM safety and security → Prompt injection and agent security
Abstract (arXiv)
As autonomous AI agents are used in regulated and safety-critical settings, organizations need effective ways to turn policy into enforceable controls. We introduce a regulatory machine learning framework that converts unstructured design artifacts (like PRDs, TDDs, and code) into verifiable runtime guardrails. Our Policy as Prompt method reads these documents and risk controls to build a source-linked policy tree. This tree is then compiled into lightweight, prompt-based classifiers for real-time runtime monitoring. The system is built to enforce least privilege and data minimization. For conformity assessment, it provides complete provenance, traceability, and audit logging, all integrated with a human-in-the-loop review process. Evaluations show our system reduces prompt-injection risk, blocks out-of-scope requests, and limits toxic outputs. It also generates auditable rationales aligned with AI governance frameworks. By treating policies as executable prompts (a policy-as-code for agents), this approach enables secure-by-design deployment, continuous compliance, and scalable AI safety and AI security assurance for regulatable ML.
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 11 of its 14 references that are in the map. First posted on arXiv 28 Sep 2025.
Builds on
11 of 14 references.
Closest ancestors · 2
Same direction, newest first
- 2025CAPTURE: Context-Aware Prompt Injection Testing and Robustness Enhancement18 May
- 2023Prompt Injection attack against LLM-integrated Applications8 Jun
Other references · 9
- 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 129
- 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?cited by 85
- 2024Building Guardrails for Large Language Modelscited by 65
- 2024TrustAgent: Towards Safe and Trustworthy LLM-based Agentscited by 64
- 2024Large Language Model Agent in Financial Trading: A Surveycited by 60
- 2025Contextual Agent Security: A Policy for Every Purposecited by 59
- 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attackscited by 49
- 2024SCULPT: Systematic Tuning of Long Promptscited by 9
- 2024AI Act and Large Language Models (LLMs): When critical issues and privacy impact require human and ethical oversightcited by 6
Built on by
Citing papers per year: 2025: 1; 2026: 13.
14 papers.
Sort:
By area:
- 2026Formal Policy Enforcement for Real-World Agentic Systemscited by 33
- 2026From Governance Norms to Enforceable Controls: A Layered Translation Method for Runtime Guardrails in Agentic AIcited by 7
- 2026Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agentscited by 5
- 2026Deontic Policies for Runtime Governance of Agentic AI Systemscited by 5
- 2025CoPE: A Small Language Model for Steerable and Scalable Content Labelingcited by 4
- 2026Think Before You Act -- A Neurocognitive Governance Model for Autonomous AI Agentscited by 2
- 2026Autoformalization of Agent Instructions into Policy-as-Codecited by 2
- 2026Dynamic Capability Scoping for Enterprise AI Agents: A Synthetic Dataset and Three-Source Permission Architecturecited by 1
- 2026Probing the Structure and Dynamics of LLM Value Expression through Value Conflictscited by 0
- 2026MetroLLM-Bench: Evaluating Language Models as Transit Kiosk Runtimescited by 0
- 2026Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governancecited by 0
- 2026Empirical Evaluation of Task-Based Permission Scoping Architecture for AI Agentscited by 0
- 2026Holding Agents Accountable with Verifiable Agent Insurance13 Oct 2026
- 2026Policy-as-Skill: Governed LLM Decision Support with Evidence, Deterministic Control, and Audit22 Sep 2026
- 2026Empirical Evaluation of Task-Based Permission Scoping Architecture for AI Agents14 Sep 2026
- 2026Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governance9 Sep 2026
- 2026MetroLLM-Bench: Evaluating Language Models as Transit Kiosk Runtimes9 Sep 2026
- 2026Probing the Structure and Dynamics of LLM Value Expression through Value Conflicts7 Sep 2026
- 2026Dynamic Capability Scoping for Enterprise AI Agents: A Synthetic Dataset and Three-Source Permission Architecture24 Jul 2026
- 2026Autoformalization of Agent Instructions into Policy-as-Code25 Jun 2026
- 2026Deontic Policies for Runtime Governance of Agentic AI Systems17 Jun 2026
- 2026Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents11 Jun 2026
- 2026Think Before You Act -- A Neurocognitive Governance Model for Autonomous AI Agents28 Apr 2026
- 2026From Governance Norms to Enforceable Controls: A Layered Translation Method for Runtime Guardrails in Agentic AI6 Apr 2026
2 more
- 2026Formal Policy Enforcement for Real-World Agentic Systemscited by 33
- 2026From Governance Norms to Enforceable Controls: A Layered Translation Method for Runtime Guardrails in Agentic AIcited by 7
- 2026Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agentscited by 5
- 2026Deontic Policies for Runtime Governance of Agentic AI Systemscited by 5
- 2026Autoformalization of Agent Instructions into Policy-as-Codecited by 2
- 2026Dynamic Capability Scoping for Enterprise AI Agents: A Synthetic Dataset and Three-Source Permission Architecturecited by 1
- 2026Dynamic Capability Scoping for Enterprise AI Agents: A Synthetic Dataset and Three-Source Permission Architecture24 Jul 2026
- 2026Autoformalization of Agent Instructions into Policy-as-Code25 Jun 2026
- 2026Deontic Policies for Runtime Governance of Agentic AI Systems17 Jun 2026
- 2026Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents11 Jun 2026
- 2026From Governance Norms to Enforceable Controls: A Layered Translation Method for Runtime Guardrails in Agentic AI6 Apr 2026
- 2026Formal Policy Enforcement for Real-World Agentic Systems18 Feb 2026
- 2025CoPE: A Small Language Model for Steerable and Scalable Content Labelingcited by 4
- 2026Probing the Structure and Dynamics of LLM Value Expression through Value Conflictscited by 0
- 2026MetroLLM-Bench: Evaluating Language Models as Transit Kiosk Runtimescited by 0
- 2026Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governancecited by 0
- 2026Empirical Evaluation of Task-Based Permission Scoping Architecture for AI Agentscited by 0
- 2026Policy-as-Skill: Governed LLM Decision Support with Evidence, Deterministic Control, and Auditcited by 0
- 2026Holding Agents Accountable with Verifiable Agent Insurancecited by 0
- 2026Holding Agents Accountable with Verifiable Agent Insurance13 Oct 2026
- 2026Policy-as-Skill: Governed LLM Decision Support with Evidence, Deterministic Control, and Audit22 Sep 2026
- 2026Empirical Evaluation of Task-Based Permission Scoping Architecture for AI Agents14 Sep 2026
- 2026Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governance9 Sep 2026
- 2026MetroLLM-Bench: Evaluating Language Models as Transit Kiosk Runtimes9 Sep 2026
- 2026Probing the Structure and Dynamics of LLM Value Expression through Value Conflicts7 Sep 2026
- 2025CoPE: A Small Language Model for Steerable and Scalable Content Labeling19 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2025Executable Governance for AI: Translating Policies into Rules Using LLMs
- 2025Learning Efficient Guardrails for Compliance
- 2026AdaGuard: Enhancing Safety and Policy Compliance with Reasoning-Enabled LLM-As-A-Judge Guardrails
- 2026LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails
- 2026PolicyGuard: Prompt-Configurable Semantic DLP for LLM Coding Agents
- 2026Don't Make Models Guess Security and Safety: Symbolic Guardrails for Domain-Specific AI Agents
- 2026Runtime Governance for AI Agents: Policies on Paths
- 2026SingGuard-NSFA: Extensible Guardrails for Agentic AI via Generative Reasoning and Real-Time Classification