Mechanical Enforcement for LLM Governance:Evidence of Governance-Task Decoupling in Financial Decision Systems
Not on the map. Why?
Abstract (arXiv)
Large language models in regulated financial workflows are governed by natural-language policies that the same model interprets, creating a principal--agent failure: outputs can appear compliant without being compliant. Existing evaluation measures task accuracy but not whether governance constrains behaviour at the decision rationale level -- where regulated decisions must be auditable. We introduce five governance metrics that quantify policy compliance at the rationale level and apply them in a synthetic banking domain to compare text-only governance against mechanical enforcement: four primitives operating outside the model's interpretive loop. Under text-only governance, 27% of deferrals carry no decision-relevant information. Mechanical enforcement reduces this rate by 73%, more than doubles deferral information content, and raises task accuracy from MCC~0.43 to 0.88. The improvement is driven by architectural separation: LLM-generated rationales under mechanical enforcement show comparable CDL to text-only governance -- the gain comes from removing clear-cut decisions from the model's control. A causal ablation confirms that each primitive is individually necessary. Our central finding is a governance-task decoupling: under structural stress, text-only governance degrades on both dimensions simultaneously, whereas mechanical enforcement preserves governance quality even as task performance drops. This implies that governance and task evaluation are distinct axes: accuracy is not a sufficient proxy for governance in regulated AI systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 37 references that are in the map. First posted on arXiv 14 May 2026.
Builds on
19 of 37 references.
References · 19
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2016Concrete Problems in AI Safetycited by 1,582
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2022Red Teaming Language Models with Language Modelscited by 983
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
- 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
- 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
- 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
- 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 140
- 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
- 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
- 2021Machine Learning with a Reject Option: A surveycited by 107
7 more
- 2023Who Should Predict? Exact Algorithms For Learning to Defer to Humanscited by 68
- 2023Realistic Synthetic Financial Transactions for Anti-Money Laundering Modelscited by 47
- 2023Goodhart's Law in Reinforcement Learningcited by 37
- 2026Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphscited by 17
- 2025The Agentic Regulator: Risks for AI in Finance and a Proposed Agent-based Framework for Governancecited by 3
- 2025Model Risk Management for Generative AI In Financial Institutionscited by 2
- 2026Towards Selection as Power: Bounding Decision Authority in Autonomous Agentscited by 2
Built on by
Not cited yet.