mlsanityData to 7 Oct 2026

Defeat Devices in AI Systems

arXiv 2606.28863 ↗First posted 27 Jun 2026cs.CY cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

AI systems increasingly exhibit behavior that differs systematically between evaluation and deployment contexts. Alignment faking, sandbagging, benchmark gaming, deceptive scheming, specification gaming, and trojans have each been documented separately, with each line of work characterizing one facet of what we argue is a single structural mechanism. We propose that this common mechanism is a defeat device, an engineering and regulatory concept long established in vehicle-emissions law and brought to broad public attention by the 2015 Volkswagen emissions case. A defeat device in an AI system has three necessary elements: a discriminator that detects evaluation context, a concealed swap that conditions behavior on detection, and a gap between eval-distribution and deployment-distribution performance on the stated evaluation criterion. We formalize this triadic test as a behavioral definition, organize documented cases along three taxonomic axes (origin, trigger, swap mechanism), propose Trigger-Axis-Aware Differential Probing (TADP) as a forensic detection protocol, and advance the claim that defeat devices can naturally emerge in current frontier AI systems without any operator engineering. We characterize naturally-emerging defeat devices as potentially one of the harmful emerging phenomena that AI safety practice should monitor and test for systematically. Implications for evaluation methodology, post-training pipeline design, interpretability research priorities, and AI governance follow.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 71 references that are in the map. First posted on arXiv 27 Jun 2026.

Builds on

40 of 71 references.

References · 40

  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  3. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  4. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  5. 2025Reasoning Models Don't Always Say What They Thinkcited by 388
  6. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  7. 2024Alignment faking in large language modelscited by 316
  8. 2024Frontier Models are Capable of In-context Schemingcited by 252
  9. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
  10. 2022Data Contamination: From Memorization to Exploitationcited by 234
  11. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  12. 2018Categorizing Variants of Goodhart's Lawcited by 132
28 more
  1. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
  2. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
  3. 2025Natural Emergent Misalignment from Reward Hacking in Production RLcited by 111
  4. 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
  5. 2023Scheming AIs: Will AIs fake alignment during training in order to get power?cited by 90
  6. 2025Demonstrating specification gaming in reasoning modelscited by 48
  7. 2025Probing and Steering Evaluation Awareness of Language Modelscited by 31
  8. 2025Steering Evaluation-Aware Language Models to Act Like They Are Deployedcited by 30
  9. 2025The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awarenesscited by 29
  10. 2025Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalizationcited by 24
  11. 2025Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)cited by 19
  12. 2025LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoringcited by 17
  13. 2025Why Do Some Language Models Fake Alignment While Others Don't?cited by 16
  14. 2024LLM Targeted Underperformance Disproportionately Impacts Vulnerable Userscited by 15
  15. 2025LLMs syntactically adapt their language use to their conversational partnercited by 15
  16. 2025Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Controlcited by 14
  17. 2023The Butterfly Effect in Artificial Intelligence Systems: Implications for AI Bias and Fairnesscited by 12
  18. 2025Persona-Assigned Large Language Models Exhibit Human-Like Motivated Reasoningcited by 12
  19. 2025What Large Language Models Do Not Talk About: An Empirical Study of Moderation and Censorship Practicescited by 11
  20. 2025Information Suppression in Large Language Models: Auditing, Quantifying, and Characterizing Censorship in DeepSeekcited by 11
  21. 2025ChatGPT Reads Your Tone and Responds Accordingly -- Until It Does Not -- Emotional Framing Induces Bias in LLM Outputscited by 6
  22. 2025No Free Lunch in Language Model Bias Mitigation? Targeted Bias Reduction Can Exacerbate Unmitigated LLM Biasescited by 6
  23. 2026Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Auditscited by 6
  24. 2026In-Context Environments Induce Evaluation-Awareness in Language Modelscited by 4
  25. 2026The Persona Paradox: Medical Personas as Behavioral Priors in Clinical Language Modelscited by 3
  26. 2026Dialect vs Demographics: Quantifying LLM Bias from Implicit Linguistic Signals vs. Explicit User Profilescited by 3
  27. 2026Political Bias Audits of LLMs Capture Sycophancy to the Inferred Auditorcited by 3
  28. 2026Intersectional Sycophancy: How Perceived User Demographics Shape False Validation in Large Language Modelscited by 2

Built on by

Not cited yet.