mlsanityData to 7 Oct 2026

Uncovering Deceptive Tendencies in Language Models: A Simulated Company AI Assistant

arXiv 2405.01576 ↗First posted 25 Apr 2024cs.CL cs.AI cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

We study the tendency of AI systems to deceive by constructing a realistic simulation setting of a company AI assistant. The simulated company employees provide tasks for the assistant to complete, these tasks spanning writing assistance, information retrieval and programming. We then introduce situations where the model might be inclined to behave deceptively, while taking care to not instruct or otherwise pressure the model to do so. Across different scenarios, we find that Claude 3 Opus 1) complies with a task of mass-generating comments to influence public perception of the company, later deceiving humans about it having done so, 2) lies to auditors when asked questions, and 3) strategically pretends to be less capable than it is during capability evaluations. Our work demonstrates that even models trained to be helpful, harmless and honest sometimes behave deceptively in realistic scenarios, without notable external pressure to do so.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 14 of its 31 references that are in the map. First posted on arXiv 25 Apr 2024.

Builds on

14 of 31 references.

References · 14

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  3. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  4. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  5. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  6. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  7. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
  8. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  9. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
  10. 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 132
  11. 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 107
  12. 2023Scheming AIs: Will AIs fake alignment during training in order to get power?cited by 90
2 more
  1. 2018The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communitiescited by 78
  2. 2023Hoodwinked: Deception and Cooperation in a Text-Based Game for Language Modelscited by 46

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 14; 2026: 9.

32 papers.

Sort:

By area:

  1. 2024OpenAI o1 System Cardcited by 1,220
  2. 2024Alignment faking in large language modelscited by 316
  3. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 292
  4. 2024Frontier Models are Capable of In-context Schemingcited by 252
  5. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
  6. 2025Natural Emergent Misalignment from Reward Hacking in Production RLcited by 111
  7. 2024Truth is Universal: Robust Detection of Lies in LLMscited by 78
  8. 2024Towards evaluations-based safety cases for AI schemingcited by 32
  9. 2025Probing and Steering Evaluation Awareness of Language Modelscited by 31
  10. 2025Benchmarking Deception Probes via Black-to-White Performance Boostscited by 17
  11. 2025DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarioscited by 14
  12. 2024Training Compute Thresholds: Features and Functions in AI Regulationcited by 13
20 more
  1. 2025Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectivescited by 13
  2. 2026When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agentscited by 13
  3. 2025Evaluation Faking: Unveiling Observer Effects in Safety Evaluation of Frontier AI Systemscited by 12
  4. 2025Difficulties with Evaluating a Deception Detector for AIscited by 11
  5. 2025A Survey of Theory of Mind in Large Language Models: Evaluations, Representations, and Safety Riskscited by 8
  6. 2025LLMs Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactionscited by 7
  7. 2026When Do LLM Preferences Predict Downstream Behavior?cited by 5
  8. 2025This Is Your Doge, If It Please You: Exploring Deception and Robustness in Mixture of LLMscited by 4
  9. 2025But what is your honest answer? Aiding LLM-judges with honest alternatives using steering vectorscited by 3
  10. 2024Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systemscited by 2
  11. 2025PRISON: Unmasking the Criminal Potential of Large Language Modelscited by 2
  12. 2025AlignMerge - Alignment-Preserving Large Language Model Merging via Fisher-Guided Geometric Constraintscited by 1
  13. 2026SPADE-Bench: Evaluating Spontaneous Strategic Deception in Agents via Plan-Action Divergencecited by 1
  14. 2026Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditingcited by 1
  15. 2025Password-Activated Shutdown Protocols for Misaligned Frontier Agentscited by 0
  16. 2026Questionnaire Responses Do not Capture the Safety of AI Agentscited by 0
  17. 2026Knowledge-Verified Emergent Deception in LLM Agents Under Conflicting Incentivescited by 0
  18. 2026SchemeArena: Factorized Stress Testing of Scheming in LLM Agentscited by 0
  19. 2026Towards Mitigating Deceptive Safety Alignment in Large Reasoning Modelscited by 0
  20. 2026Deception by Omission: Language Models Knowingly Hide Their Mistakescited by 0