# Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems

arXiv: https://arxiv.org/abs/2610.12313 · HTML: https://mlsanity.com/paper/2610.12313/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Builds on (most cited first, 14 of 14)

- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Qwen3Guard Technical Report](https://mlsanity.com/paper/2510.14276/) (2025)
- [GraphCompliance: Structure-Aware Regulatory Compliance through Policy-Context Graph Alignment](https://mlsanity.com/paper/2510.26309/) (2025)
- [LegiLM: A Fine-Tuned Legal Language Model for Data Compliance](https://mlsanity.com/paper/2409.13721/) (2024)
- [Training-Free Policy Violation Detection via Activation-Space Whitening in LLMs](https://mlsanity.com/paper/2512.03994/) (2025)
- [LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails](https://mlsanity.com/paper/2605.17329/) (2026)
- [CourtGuard: A Model-Agnostic Framework for Zero-Shot Policy Adaptation in LLM Safety](https://mlsanity.com/paper/2602.22557/) (2026)
- [ComplianceNLP: Knowledge-Graph-Augmented RAG for Multi-Framework Regulatory Gap Detection](https://mlsanity.com/paper/2604.23585/) (2026)
- [RuleSafe-VL: Evaluating Rule-Conditioned Decision Reasoning in Vision-Language Content Moderation](https://mlsanity.com/paper/2605.07760/) (2026)
- [FinGuard: Detecting Financial Regulatory Non-Compliance in LLM Interactions](https://mlsanity.com/paper/2605.29427/) (2026)
- [Citation-Closure Retrieval and Per-Rule Attribution for Real-World Regulatory Compliance Question Answering](https://mlsanity.com/paper/2605.29742/) (2026)
- [The Refusal Residue: When Probes Catch Alignment Faking and When They Don't](https://mlsanity.com/paper/2607.13346/) (2026)
- [What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models](https://mlsanity.com/paper/2608.16852/) (2026)

## Abstract (arXiv)

Large language model compliance systems are deployed on the assumption that a verdict depends on the regulatory rule it is given. We test this directly across five models and 20 regulatory and platform-policy domains: delete, swap, or negate the governing rule while holding the case fixed, and check whether the verdict changes (OCS) or the model's internal representation of compliance shifts at all (ICS-delta). Neither moves much: models' verdicts are often invariant to substantial perturbations of the supplied rule, and the guard model, evaluated here under a custom-rule adaptation of its native taxonomy, is the least rule-sensitive and least accurate of the five, barely above chance (51%, versus 90-92% for general-purpose models). This reflects easy cases more than blanket neglect: on cases where deleting the rule changes a previously correct model prediction, models do track it closely. Neither better prompting nor direct intervention on the model's internal representations closes this gap. Accuracy alone does not establish that a compliance verdict is grounded in the supplied rule.