# Policy-as-Prompt: Turning AI Governance Rules into Guardrails for AI Agents

arXiv: https://arxiv.org/abs/2509.23994 · HTML: https://mlsanity.com/paper/2509.23994/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 11 of its 14 references that are in the map.
- First posted on arXiv 28 Sep 2025.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 11 of 11)

- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- [ProPILE: Probing Privacy Leakage in Large Language Models](https://mlsanity.com/paper/2307.01881/) (2023)
- [A Survey of LLM-based Agents in Medicine: How far are we from Baymax?](https://mlsanity.com/paper/2502.11211/) (2025)
- [Building Guardrails for Large Language Models](https://mlsanity.com/paper/2402.01822/) (2024)
- [TrustAgent: Towards Safe and Trustworthy LLM-based Agents](https://mlsanity.com/paper/2402.01586/) (2024)
- [Large Language Model Agent in Financial Trading: A Survey](https://mlsanity.com/paper/2408.06361/) (2024)
- [Contextual Agent Security: A Policy for Every Purpose](https://mlsanity.com/paper/2501.17070/) (2025)
- [Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks](https://mlsanity.com/paper/2502.08586/) (2025)
- [SCULPT: Systematic Tuning of Long Prompts](https://mlsanity.com/paper/2410.20788/) (2024)
- [AI Act and Large Language Models (LLMs): When critical issues and privacy impact require human and ethical oversight](https://mlsanity.com/paper/2404.00600/) (2024)
- [CAPTURE: Context-Aware Prompt Injection Testing and Robustness Enhancement](https://mlsanity.com/paper/2505.12368/) (2025)

## Built on by (most cited first, 14 of 14)

- [Formal Policy Enforcement for Real-World Agentic Systems](https://mlsanity.com/paper/2602.16708/) (2026)
- [From Governance Norms to Enforceable Controls: A Layered Translation Method for Runtime Guardrails in Agentic AI](https://mlsanity.com/paper/2604.05229/) (2026)
- [Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents](https://mlsanity.com/paper/2606.13174/) (2026)
- [Deontic Policies for Runtime Governance of Agentic AI Systems](https://mlsanity.com/paper/2606.19464/) (2026)
- [CoPE: A Small Language Model for Steerable and Scalable Content Labeling](https://mlsanity.com/paper/2512.18027/) (2025)
- [Think Before You Act -- A Neurocognitive Governance Model for Autonomous AI Agents](https://mlsanity.com/paper/2604.25684/) (2026)
- [Autoformalization of Agent Instructions into Policy-as-Code](https://mlsanity.com/paper/2606.26649/) (2026)
- [Dynamic Capability Scoping for Enterprise AI Agents: A Synthetic Dataset and Three-Source Permission Architecture](https://mlsanity.com/paper/2607.22445/) (2026)
- [Probing the Structure and Dynamics of LLM Value Expression through Value Conflicts](https://mlsanity.com/paper/2609.07296/) (2026)
- [MetroLLM-Bench: Evaluating Language Models as Transit Kiosk Runtimes](https://mlsanity.com/paper/2609.10016/) (2026)
- [Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governance](https://mlsanity.com/paper/2609.10105/) (2026)
- [Empirical Evaluation of Task-Based Permission Scoping Architecture for AI Agents](https://mlsanity.com/paper/2609.15422/) (2026)
- [Policy-as-Skill: Governed LLM Decision Support with Evidence, Deterministic Control, and Audit](https://mlsanity.com/paper/2609.27087/) (2026)
- [Holding Agents Accountable with Verifiable Agent Insurance](https://mlsanity.com/paper/2610.18230/) (2026)

## Abstract (arXiv)

As autonomous AI agents are used in regulated and safety-critical settings, organizations need effective ways to turn policy into enforceable controls. We introduce a regulatory machine learning framework that converts unstructured design artifacts (like PRDs, TDDs, and code) into verifiable runtime guardrails. Our Policy as Prompt method reads these documents and risk controls to build a source-linked policy tree. This tree is then compiled into lightweight, prompt-based classifiers for real-time runtime monitoring. The system is built to enforce least privilege and data minimization. For conformity assessment, it provides complete provenance, traceability, and audit logging, all integrated with a human-in-the-loop review process. Evaluations show our system reduces prompt-injection risk, blocks out-of-scope requests, and limits toxic outputs. It also generates auditable rationales aligned with AI governance frameworks. By treating policies as executable prompts (a policy-as-code for agents), this approach enables secure-by-design deployment, continuous compliance, and scalable AI safety and AI security assurance for regulatable ML.