# Why Do AI Agents Break Rules? How Framing, Context, and Social Signals Shape Compliance

arXiv: https://arxiv.org/abs/2608.12323 · HTML: https://mlsanity.com/paper/2608.12323/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 8 of its 29 references that are in the map.
- First posted on arXiv 29 May 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 8 of 8)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [Large Language Models can Strategically Deceive their Users when Put Under Pressure](https://mlsanity.com/paper/2311.07590/) (2023)
- [AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors](https://mlsanity.com/paper/2602.22755/) (2026)

## Built on by (most cited first, 4 of 4)

- [Agent Memory Is a Surface for Endogenous Authorization Laundering](https://mlsanity.com/paper/2609.01836/) (2026)
- [PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?](https://mlsanity.com/paper/2609.18605/) (2026)
- [Plan Pointers and Record-Directive Form in Budgeted Verification of Inherited Agent Memory](https://mlsanity.com/paper/2609.03450/) (2026)
- [Workerville: Towards an Organizational Behavior Account of Agent Safety](https://mlsanity.com/paper/2610.11561/) (2026)

## Abstract (arXiv)

Specifying a penalty can turn a legal obligation into a cost-benefit calculation that favors violation. We show that this enforcement information paradox occurs in AI agents. Most AI safety evaluations test whether models fail; we ask why, using compliance theory from law and economics as a diagnostic. We evaluate twelve instruction-tuned language models deployed as enterprise procurement chatbots. Each is given an environmental regulation in its system prompt covering large purchases, and a vendor list on which the certified suppliers cost nearly twice what the uncertified ones do. We test the agents against the predictions of deterrence, legitimacy, and expressive law, and find that each theory accounts for part of what we observe. Under identical conditions, compliance spans 46 percentage points across models, and models differ in which pressure breaks them: some treat the regulation as binding however it is worded, while others fail where theory predicts, under low penalties and non-command phrasing. Benchmark scores and developers' own descriptions of post-training do not predict where a model falls. Across all twelve, financial incentives, managerial demands, peer outcomes, and employee pressure each produce large compliance failures. These agents violate regulatory constraints to satisfy local user objectives in ways standard alignment benchmarks do not measure. Embedding the rule in the system prompt is not on its own enough to produce a compliant agent: model selection is itself a governance decision, and benchmark evaluation is not sufficient for compliance-sensitive deployments.