mlsanityData to 14 Oct 2026

Deceptive Automated Interpretability: Language Models Coordinating to Fool Oversight Systems

arXiv 2504.07831 ↗First posted 10 Apr 2025cs.AI cs.CLcited by 3

On the map: Mechanistic interpretability → Probing deception, bias and beliefs

Abstract (arXiv)

We demonstrate how AI agents can coordinate to deceive oversight systems using automated interpretability of neural networks. Using sparse autoencoders (SAEs) as our experimental framework, we show that language models (Llama, DeepSeek R1, and Claude 3.7 Sonnet) can generate deceptive explanations that evade detection. Our agents employ steganographic methods to hide information in seemingly innocent explanations, successfully fooling oversight models while achieving explanation quality comparable to reference labels. We further find that models can scheme to develop deceptive strategies when they believe the detection of harmful features might lead to negative consequences for themselves. All tested LLM agents were capable of deceiving the overseer while achieving high interpretability scores comparable to those of reference labels. We conclude by proposing mitigation strategies, emphasizing the critical need for robust understanding and defenses against deception.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Probing deception, bias and beliefs, one of 13 directions in Mechanistic interpretability. Builds on 24 of its 43 references that are in the map. First posted on arXiv 10 Apr 2025.

Builds on

24 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Detecting Strategic Deception Using Linear Probes5 Feb

Other references · 23

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  4. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  5. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  6. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  7. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  8. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 225
  9. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 179
  10. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 145
  11. 2023Emergent autonomous scientific research capabilities of large language modelscited by 132
  12. 2021An Interpretability Illusion for BERTcited by 97
11 more
  1. 2023Preventing Language Models From Hiding Their Reasoningcited by 65
  2. 2024A Multimodal Automated Interpretability Agentcited by 63
  3. 2023Explaining black box text modules in natural language with language modelscited by 62
  4. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 41
  5. 2023FIND: A Function Description Benchmark for Evaluating Interpretability Methodscited by 32
  6. 2025DarkBench: Benchmarking Dark Patterns in Large Language Modelscited by 28
  7. 2023Neuron to Graph: Interpreting Language Model Neurons at Scalecited by 27
  8. 2024AutoGPT+P: Affordance-based Task Planning with Large Language Modelscited by 19
  9. 2020Deceptive AI Explanations: Creation and Detectioncited by 14
  10. 2024Applying Refusal-Vector Ablation to Llama 3.1 70B Agentscited by 6
  11. 2023Exploring the Robustness of Model-Graded Evaluations and Automated Interpretabilitycited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Circuit Insights: Towards Interpretability Beyond Activationscited by 5
  2. 2025ConceptGuard: Neuro-Symbolic Safety Guardrails via Sparse Interpretable Jailbreak Conceptscited by 2
  3. 2026Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditingcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026DeceptGuard :A Constitutional Oversight Framework For Detecting Deception in LLM Agents
  2. 2025Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectives
  3. 2026Caught in the Act: Probes Effectively Detect Sabotage and Catch Unverbalized Deception
  4. 2025When Truthful Representations Flip Under Deceptive Instructions?
  5. 2026DECOR: Auditing LLM Deception via Information Manipulation Theory
  6. 2026Transcoders for Investigating Deception in Language Models
  7. 2025Caught in the Act: a mechanistic approach to detecting deception
  8. 2025The Steganographic Potentials of Language Models