The Compliance Trap: How Structural Constraints Degrade Frontier AI Metacognition Under Adversarial Pressure
Not on the map. Why?
Abstract (arXiv)
As frontier AI models are deployed in high-stakes decision pipelines, their ability to maintain metacognitive stability (knowing what they do not know, detecting errors, seeking clarification) under adversarial pressure is a critical safety requirement. Current safety evaluations focus on detecting strategic deception (scheming); we investigate a more fundamental failure mode: cognitive collapse. We present SCHEMA, an evaluation of 11 frontier models from 8 vendors across 67,221 scored records using a 6-condition factorial design with dual-classifier scoring. We find that 8 of 11 models suffer catastrophic metacognitive degradation under adversarial pressure, with accuracy dropping by up to 30.2 percentage points (all p < 2 × 10-8, surviving Bonferroni correction). Crucially, we identify a "Compliance Trap": through factorial isolation and a benign distraction control, we demonstrate that collapse is driven not by the psychological content of survival threats, but by compliance-forcing instructions that override epistemic boundaries. Removing the compliance suffix restores performance even under active threat. Models with advanced reasoning capabilities exhibit the most severe absolute degradation, while Anthropic's Constitutional AI demonstrates near-perfect immunity. This immunity does not stem from superior capability (Google's Gemini matches its baseline accuracy) but from alignment-specific training. We release the complete dataset and evaluation infrastructure.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 3 of its 11 references that are in the map. First posted on arXiv 4 May 2026.
Builds on
3 of 11 references.
References · 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ARREST: Adversarial Resilient Regulation Enhancing Safety and Truth in Large Language Models
- 2026When Evaluation Becomes a Side Channel: Regime Leakage and Structural Mitigations for Alignment Assessment
- 2025Stress Testing Deliberative Alignment for Anti-Scheming Training
- 2026Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs
- 2026Does Deeper Reasoning Compromise Alignment? Revealing and Mitigating of Alignment Collapse in Large Reasoning Models
- 2026Epistemic Traps: Rational Misalignment Driven by Model Misspecification
- 2026Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack
- 2026Dual-Adversarial Safety Alignment: Cultivating Intrinsic Threat Comprehension in LRMs