mlsanityData to 14 Oct 2026

Honesty Is the Best Policy: Defining and Mitigating AI Deception

arXiv 2312.01350 ↗First posted 3 Dec 2023cs.AIcited by 40

Not on the map. Why?

Abstract (arXiv)

Deceptive agents are a challenge for the safety, trustworthiness, and cooperation of AI systems. We focus on the problem that agents might deceive in order to achieve their goals (for instance, in our experiments with language models, the goal of being evaluated as truthful). There are a number of existing definitions of deception in the literature on game theory and symbolic AI, but there is no overarching theory of deception for learning agents in games. We introduce a formal definition of deception in structural causal games, grounded in the philosophy literature, and applicable to real-world machine learning systems. Several examples and results illustrate that our formal definition aligns with the philosophical and commonsense meaning of deception. Our main technical result is to provide graphical criteria for deception. We show, experimentally, that these results can be used to mitigate deception in reinforcement learning agents and language models.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Builds on 40 of its 128 references that are in the map. First posted on arXiv 3 Dec 2023.

Builds on

40 of 128 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  6. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  9. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  10. 2022In-context Learning and Induction Headscited by 964
  11. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  12. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
28 more
  1. 2017Certified Defenses for Data Poisoning Attackscited by 303
  2. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
  3. 2023An Overview of Catastrophic AI Riskscited by 222
  4. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 209
  5. 2018Do GANs leave artificial fingerprints?cited by 190
  6. 2022Talking About Large Language Modelscited by 175
  7. 2020Open Problems in Cooperative AIcited by 168
  8. 2021Alignment of Language Agentscited by 168
  9. 2023Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigationscited by 158
  10. 2023Taken out of context: On measuring situational awareness in LLMscited by 143
  11. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 131
  12. 2021Truthful AI: Developing and governing AI that does not liecited by 113
  13. 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 108
  14. 2022Is Power-Seeking AI an Existential Risk?cited by 105
  15. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 83
  16. 2023Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblockscited by 78
  17. 2021Agent Incentives: A Causal Perspectivecited by 51
  18. 2023Characterizing Manipulation from AI Systemscited by 43
  19. 2016The AGI Containment Problemcited by 36
  20. 2018Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibilitycited by 32
  21. 2022Discovering Agentscited by 28
  22. 2022Path-Specific Objectives for Safer Agent Incentivescited by 20
  23. 2023Reasoning about Causality in Gamescited by 20
  24. 2021Equilibrium Refinements for Multi-Agent Influence Diagrams: Theory and Practicecited by 9
  25. 2021Definitions of intent suitable for algorithmscited by 8
  26. 2018Unpacking the Social Media Bot: A Typology to Guide Research and Policycited by 7
  27. 2020Studying Dishonest Intentions in Brazilian Portuguese Textscited by 2
  28. 2021Extending counterfactual accounts of intent to include oblique intentcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 17; 2025: 14; 2026: 9.

40 papers.

Sort:

By area:

  1. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 145
  2. 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganographycited by 113
  3. 2024Robust agents learn causal world modelscited by 81
  4. 2024AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agentscited by 31
  5. 2024Standards for Belief Representations in LLMscited by 30
  6. 2024The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Modelscited by 29
  7. 2024The Reasons that Agents Act: Intention and Instrumental Goalscited by 23
  8. 2024A Survey on the Honesty of Large Language Modelscited by 23
  9. 2025General agents contain world modelscited by 17
  10. 2024From Imitation to Introspection: Probing Self-Consciousness in Language Modelscited by 14
  11. 2025Difficulties with Evaluating a Deception Detector for AIscited by 13
  12. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systemscited by 12
28 more
  1. 2024Language Models can Subtly Deceive Without Lying: A Case Study on Strategic Phrasing in Legislationcited by 10
  2. 2025The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blindcited by 8
  3. 2024Measuring Goal-Directednesscited by 7
  4. 2025Collaborate, Deliberate, Evaluate: How LLM Alignment Affects Coordinated Multi-Agent Outcomescited by 7
  5. 2025LH-Deception: Simulating and Understanding LLM Deceptive Behaviors in Long-Horizon Interactionscited by 7
  6. 2024Towards Safe and Honest AI Agents with Neural Self-Other Overlapcited by 6
  7. 2024An Assessment of Model-On-Model Deceptioncited by 5
  8. 2025OpenDeception: Learning Deception and Trust in Human-AI Interaction via Multi-Agent Simulationcited by 5
  9. 2025Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Riskscited by 5
  10. 2025Enhancing LLM Agent Safety via Causal Influence Promptingcited by 5
  11. 2025Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Promptscited by 5
  12. 2024Observation Interference in Partially Observable Assistance Gamescited by 4
  13. 2024Possible Principles for Aligned Structure Learning Agentscited by 3
  14. 2025A Principle of Targeted Intervention for Multi-Agent Reinforcement Learningcited by 3
  15. 2025PRISON: Unmasking the Criminal Potential of Large Language Modelscited by 2
  16. 2025Learning "Partner-Aware" Collaborators in Multi-Party Collaborationcited by 2
  17. 2026Persuasion Attacks Can Decrease Effectiveness of CoT Monitoringcited by 2
  18. 2026Knowledge-Verified Emergent Deception in LLM Agents Under Conflicting Incentivescited by 2
  19. 2024Attaining Human`s Desirable Outcomes in Human-AI Interaction via Structural Causal Gamescited by 1
  20. 2025Higher-Order Belief in Incomplete Information MAIDscited by 1
  21. 2026SPADE-Bench: Evaluating Spontaneous Strategic Deception in Agents via Plan-Action Divergencecited by 1
  22. 2026The Impossibility of Eliciting Latent Knowledgecited by 1
  23. 2024Evaluating Language Model Character Traitscited by 0
  24. 2026Intentional Deception as Controllable Capability in LLM Agentscited by 0
  25. 2026DeceptGuard :A Constitutional Oversight Framework For Detecting Deception in LLM Agentscited by 0
  26. 2026A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracingcited by 0
  27. 2026When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Gamescited by 0
  28. 2026Flout at Your Own Risk: LLMs Struggle with Pragmatic Cooperativity Under Epistemic Asymmetrycited by 0