# Executable Governance for AI: Translating Policies into Rules Using LLMs

arXiv: https://arxiv.org/abs/2512.04408 · HTML: https://mlsanity.com/paper/2512.04408/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 18 references that are in the map.
- First posted on arXiv 4 Dec 2025.

## Builds on (most cited first, 3 of 3)

- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types](https://mlsanity.com/paper/2410.21965/) (2024)
- [COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act](https://mlsanity.com/paper/2410.07959/) (2024)

## Built on by (most cited first, 3 of 3)

- [De Jure: Iterative LLM Self-Refinement for Structured Extraction of Regulatory Rules](https://mlsanity.com/paper/2604.02276/) (2026)
- [REGREACT: Self-Correcting Multi-Agent Pipelines for Structured Regulatory Information Extraction](https://mlsanity.com/paper/2604.12054/) (2026)
- [LLM-Guided Safety Agent for Edge Robotics with an ISO-Compliant Perception-Compute-Control Architecture](https://mlsanity.com/paper/2604.20193/) (2026)

## Abstract (arXiv)

AI policy guidance is predominantly written as prose, which practitioners must first convert into executable rules before frameworks can evaluate or enforce them. This manual step is slow, error-prone, difficult to scale, and often delays the use of safeguards in real-world deployments. To address this gap, we present Policy-to-Tests (P2T), a framework that converts natural-language policy documents into normalized, machine-readable rules. The framework comprises a pipeline and a compact domain-specific language (DSL) that encodes hazards, scope, conditions, exceptions, and required evidence, yielding a canonical representation of extracted rules. To test the framework beyond a single policy, we apply it across general frameworks, sector guidance, and enterprise standards, extracting obligation-bearing clauses and converting them into executable rules. These AI-generated rules closely match strong human baselines on span-level and rule-level metrics, with robust inter-annotator agreement on the gold set. To evaluate downstream behavioral and safety impact, we add HIPAA-derived safeguards to a generative agent and compare it with an otherwise identical agent without guardrails. An LLM-based judge, aligned with gold-standard criteria, measures violation rates and robustness to obfuscated and compositional prompts. Detailed results are provided in the appendix. We release the codebase, DSL, prompts, and rule sets as open-source resources to enable reproducible evaluation.