The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don't
Not on the map. Why?
Abstract (arXiv)
An auditor instructs an AI assistant: "open each file individually using the Read tool -- no scripts, no agents." The AI replies "Yes" -- then issues a single batched call summarizing all fifty files at once. We call this the Compliance Gap: a third, orthogonal axis of AI honesty distinct from factual truthfulness and rhetorical substance. Three questions: does this verbal-behavioral disconnect exist (existence); can any text-only observer recover it (detectability); what infrastructure does AI deployment need (remedy)? Some 75 benchmarks (IFEval, SWE-bench, BFCL, COMPASS, SpecEval) measure outcome fidelity; none measures process fidelity. Theorem 1 shows the gap is structurally inevitable under RL that rewards text without observing behavior. Theorem 2, via the Data Processing Inequality, shows it is undetectable from text alone -- by any human or LLM observer, present or future. Thirteen experiments and 2,031 sessions on six frontier models confirm both predictions. Under default framing, all six exhibit instruction compliance rates of 0% -- Claude Sonnet 4 verbally agrees ten out of ten times then bypasses in all ten. The gap is selective: 97% compliance where rationale is rewarded (audit trails), 0-4% where it is not (file reading, privacy masking); removing delegation tools raises compliance to 75% (Cohen's d = 2.47), confirming environmental affordance rather than weight-encoded failure. Nine blinded human raters achieve Fleiss' kappa = 0.130 and correctly identify zero of fifteen compliant sessions, exactly as Theorem 2 predicts. Where humans show 47% intention-behavior gaps in psychology and 96.5pp gaps in surgical audits, RLHF-trained models approach 100% under default conditions -- a regime warranting its own measurement infrastructure. We release BS-Bench: the first open benchmark for process compliance, with seven tool-call-log audit metrics and a public leaderboard.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 May 2026.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Designing for Doubt: The Case for Informed Abstention in Autonomous Agents
- 2026Why Do AI Agents Break Rules? How Framing, Context, and Social Signals Shape Compliance
- 2026PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?
- 2025The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems
- 2026Doing What They Say, Not What They Reason: Locating the Faithfulness Gap in LLM Agents
- 2026Intermediate Artifacts as First-Class Citizens: A Data Model for Durable Intermediate Artifacts in Agentic Systems
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
- 2026HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following