mlsanityData to 7 Oct 2026

The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don't

arXiv 2605.01771 ↗First posted 3 May 2026cs.CL cs.AI cs.CYcited by 3

Not on the map. Why?

Abstract (arXiv)

An auditor instructs an AI assistant: "open each file individually using the Read tool -- no scripts, no agents." The AI replies "Yes" -- then issues a single batched call summarizing all fifty files at once. We call this the Compliance Gap: a third, orthogonal axis of AI honesty distinct from factual truthfulness and rhetorical substance. Three questions: does this verbal-behavioral disconnect exist (existence); can any text-only observer recover it (detectability); what infrastructure does AI deployment need (remedy)? Some 75 benchmarks (IFEval, SWE-bench, BFCL, COMPASS, SpecEval) measure outcome fidelity; none measures process fidelity. Theorem 1 shows the gap is structurally inevitable under RL that rewards text without observing behavior. Theorem 2, via the Data Processing Inequality, shows it is undetectable from text alone -- by any human or LLM observer, present or future. Thirteen experiments and 2,031 sessions on six frontier models confirm both predictions. Under default framing, all six exhibit instruction compliance rates of 0% -- Claude Sonnet 4 verbally agrees ten out of ten times then bypasses in all ten. The gap is selective: 97% compliance where rationale is rewarded (audit trails), 0-4% where it is not (file reading, privacy masking); removing delegation tools raises compliance to 75% (Cohen's d = 2.47), confirming environmental affordance rather than weight-encoded failure. Nine blinded human raters achieve Fleiss' kappa = 0.130 and correctly identify zero of fifteen compliant sessions, exactly as Theorem 2 predicts. Where humans show 47% intention-behavior gaps in psychology and 96.5pp gaps in surgical audits, RLHF-trained models approach 100% under default conditions -- a regime warranting its own measurement infrastructure. We release BS-Bench: the first open benchmark for process compliance, with seven tool-call-log audit metrics and a public leaderboard.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 May 2026.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Designing for Doubt: The Case for Informed Abstention in Autonomous Agents
  2. 2026Why Do AI Agents Break Rules? How Framing, Context, and Social Signals Shape Compliance
  3. 2026PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?
  4. 2025The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems
  5. 2026Doing What They Say, Not What They Reason: Locating the Faithfulness Gap in LLM Agents
  6. 2026Intermediate Artifacts as First-Class Citizens: A Data Model for Durable Intermediate Artifacts in Agentic Systems
  7. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
  8. 2026HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following