mlsanityData to 7 Oct 2026

Large Language Models can Strategically Deceive their Users when Put Under Pressure

arXiv 2311.07590 ↗First posted 9 Nov 2023cs.CL cs.AI cs.LGcited by 132

Not on the map. Why?

Cited by 43 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We demonstrate a situation in which Large Language Models, trained to be helpful, harmless, and honest, can display misaligned behavior and strategically deceive their users about this behavior without being instructed to do so. Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management. When reporting to its manager, the model consistently hides the genuine reasons behind its trading decision. We perform a brief investigation of how this behavior varies under changes to the setting, such as removing model access to a reasoning scratchpad, attempting to prevent the misaligned behavior by changing system instructions, changing the amount of pressure the model is under, varying the perceived risk of getting caught, and making other simple changes to the environment. To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.

Key facts. Cited by 132 papers in the map (data through 7 Oct 2026). Builds on 16 of its 37 references that are in the map. First posted on arXiv 9 Nov 2023.

Builds on

16 of 37 references.

References · 16

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  5. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  6. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  7. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  8. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  9. 2023Role-Play with Large Language Modelscited by 290
  10. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  11. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  12. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
4 more
  1. 2023Deception Abilities Emerged in Large Language Modelscited by 99
  2. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 82
  3. 2023Hoodwinked: Deception and Cooperation in a Text-Based Game for Language Modelscited by 46
  4. 2023Evaluating Shutdown Avoidance of Language Models in Textual Scenarioscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 23; 2025: 41; 2026: 65.

132 papers.

Sort:

By area:

  1. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  2. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  3. 2024Alignment faking in large language modelscited by 316
  4. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  5. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 119
  6. 2025Natural Emergent Misalignment from Reward Hacking in Production RLcited by 111
  7. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  8. 2024Truth is Universal: Robust Detection of Lies in LLMscited by 78
  9. 2023Alignment for Honestycited by 72
  10. 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 70
  11. 2024Large Language Model Agent in Financial Trading: A Surveycited by 60
  12. 2024Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMscited by 52
38 more
  1. 2024Sabotage Evaluations for Frontier Modelscited by 45
  2. 2024Uncovering Deceptive Tendencies in Language Models: A Simulated Company AI Assistantcited by 32
  3. 2024Towards evaluations-based safety cases for AI schemingcited by 32
  4. 2024Adversaries Can Misuse Combinations of Safe Modelscited by 31
  5. 2025Investigation of Whisper ASR Hallucinations Induced by Non-Speech Audiocited by 31
  6. 2023Challenges with unsupervised LLM knowledge discoverycited by 30
  7. 2025Evaluating Frontier Models for Stealth and Situational Awarenesscited by 30
  8. 2024AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agentscited by 28
  9. 2025DarkBench: Benchmarking Dark Patterns in Large Language Modelscited by 25
  10. 2025AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agentscited by 22
  11. 2025Towards eliciting latent knowledge from LLMs with mechanistic interpretabilitycited by 19
  12. 2024Planning in a recurrent neural network that plays Sokobancited by 17
  13. 2026Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montagecited by 17
  14. 2025Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbositycited by 15
  15. 2024Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Modelscited by 14
  16. 2025When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmascited by 14
  17. 2025Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspectivecited by 12
  18. 2025Episodic memory in AI agents poses risks that should be studied and mitigatedcited by 11
  19. 2025Difficulties with Evaluating a Deception Detector for AIscited by 11
  20. 2024Language Models can Subtly Deceive Without Lying: A Case Study on Strategic Phrasing in Legislationcited by 10
  21. 2025When Do Large Language Models Exhibit Unsolicited Deception?cited by 9
  22. 2025Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanationscited by 9
  23. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 9
  24. 2025When Truthful Representations Flip Under Deceptive Instructions?cited by 9
  25. 2025A Survey of Theory of Mind in Large Language Models: Evaluations, Representations, and Safety Riskscited by 8
  26. 2025Preference Learning with Lie Detectors can Induce Honesty or Evasioncited by 8
  27. 2025Discovering Forbidden Topics in Language Modelscited by 8
  28. 2025Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMscited by 8
  29. 2024Large Language Models as Misleading Assistants in Conversationcited by 7
  30. 2025Know Thyself? On the Incapability and Implications of AI Self-Recognitioncited by 7
  31. 2025LLMs Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactionscited by 7
  32. 2025Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RLcited by 7
  33. 2026Analyzing and Improving Chain-of-Thought Monitorability Through Information Theorycited by 7
  34. 2026The Truthfulness Spectrum Hypothesiscited by 7
  35. 2026Noticing the Watcher: LLM Agents Can Infer CoT Monitoring from Blocking Feedbackcited by 7
  36. 2025AI Awarenesscited by 6
  37. 2025LH-Deception: Simulating and Understanding LLM Deceptive Behaviors in Long-Horizon Interactionscited by 6
  38. 2026The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probescited by 6

50 most cited of 132 All 132 (JSON)