mlsanityData to 14 Oct 2026

Truthful AI: Developing and governing AI that does not lie

arXiv 2110.06674 ↗First posted 13 Oct 2021cs.CY cs.AI cs.CLcited by 113

Not on the map. Why?

Abstract (arXiv)

In many contexts, lying -- the use of verbal falsehoods to deceive -- is harmful. While lying has traditionally been a human affair, AI systems that make sophisticated verbal statements are becoming increasingly prevalent. This raises the question of how we should limit the harm caused by AI "lies" (i.e. falsehoods that are actively selected for). Human truthfulness is governed by social norms and by laws (against defamation, perjury, and fraud). Differences between AI and humans present an opportunity to have more precise standards of truthfulness for AI, and to have these standards rise over time. This could provide significant benefits to public epistemics and the economy, and mitigate risks of worst-case AI futures. Establishing norms or laws of AI truthfulness will require significant work to: (1) identify clear truthfulness standards; (2) create institutions that can judge adherence to those standards; and (3) develop AI systems that are robustly truthful. Our initial proposals for these areas include: (1) a standard of avoiding "negligent falsehoods" (a generalisation of lies that is easier to assess); (2) institutions to evaluate AI systems before and after real-world deployment; and (3) explicitly training AI systems to be truthful via curated datasets and human interaction. A concerning possibility is that evaluation mechanisms for eventual truthfulness standards could be captured by political interests, leading to harmful censorship and propaganda. Avoiding this might take careful attention. And since the scale of AI speech acts might grow dramatically over the coming decades, early truthfulness standards might be particularly important because of the precedents they set.

Key facts. Cited by 113 papers in the map (data through 14 Oct 2026). Builds on 21 of its 92 references that are in the map. First posted on arXiv 13 Oct 2021.

Builds on

21 of 92 references.

References · 21

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2020Learning to summarize from human feedbackcited by 2,588
  3. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  4. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  5. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  6. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
  7. 2020A Unifying Review of Deep and Shallow Anomaly Detectioncited by 425
  8. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 422
  9. 2017Adversarial Examples: Attacks and Defenses for Deep Learningcited by 351
  10. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 344
  11. 2020A Simple Language Model for Task-Oriented Dialoguecited by 332
  12. 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
9 more
  1. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 209
  2. 2020Emergent Multi-Agent Communication in the Deep Learning Eracited by 170
  3. 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 167
  4. 2018Categorizing Variants of Goodhart's Lawcited by 149
  5. 2018Supervising strong learners by amplifying weak expertscited by 145
  6. 2016Research Priorities for Robust and Beneficial Artificial Intelligencecited by 74
  7. 2021Agent Incentives: A Causal Perspectivecited by 51
  8. 2018Horizon: Facebook's Open Source Applied Reinforcement Learning Platformcited by 48
  9. 2017Transparency: Motivations and Challengescited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 9; 2023: 36; 2024: 29; 2025: 23; 2026: 15.

113 papers.

Sort:

By area:

  1. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  2. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  3. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  4. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  5. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  6. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  7. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
  8. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 224
  9. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 188
  10. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  11. 2023Taken out of context: On measuring situational awareness in LLMscited by 143
  12. 2023Auditing large language models: a three-layered approachcited by 137
38 more
  1. 2023Do Language Models Know When They're Hallucinating References?cited by 116
  2. 2023Calibrated Language Models Must Hallucinatecited by 115
  3. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 114
  4. 2024Looking Inward: Language Models Can Learn About Themselves by Introspectioncited by 113
  5. 2022Broken Neural Scaling Lawscited by 109
  6. 2023How to Index Item IDs for Recommendation Foundation Modelscited by 102
  7. 2023ExpertQA: Expert-Curated Questions and Attributed Answerscited by 97
  8. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 83
  9. 2024Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluationcited by 80
  10. 2023Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblockscited by 78
  11. 2023Alignment for Honestycited by 74
  12. 2025The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systemscited by 72
  13. 2025Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIscited by 67
  14. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 66
  15. 2022Forecasting Future World Events with Neural Networkscited by 63
  16. 2022In conversation with Artificial Intelligence: aligning language models with human valuescited by 62
  17. 2023Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?cited by 62
  18. 2024Factual Confidence of LLMs: on Reliability and Robustness of Current Estimatorscited by 61
  19. 2024Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?cited by 61
  20. 2024Can AI Assistants Know What They Don't Know?cited by 50
  21. 2023Evaluating Superhuman Models with Consistency Checkscited by 47
  22. 2023Honesty Is the Best Policy: Defining and Mitigating AI Deceptioncited by 40
  23. 2025The Traitors: Deception and Trust in Multi-Agent Language Model Simulationscited by 34
  24. 2023Artificial Influence: An Analysis Of AI-Driven Persuasioncited by 33
  25. 2023Evaluating Hallucinations in Chinese Large Language Modelscited by 33
  26. 2023Challenges with unsupervised LLM knowledge discoverycited by 31
  27. 2024AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agentscited by 31
  28. 2024Standards for Belief Representations in LLMscited by 30
  29. 2024Logically Consistent Language Models via Neuro-Symbolic Integrationcited by 30
  30. 2024Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluationcited by 29
  31. 2025SciSciGPT: Advancing Human-AI Collaboration in the Science of Sciencecited by 27
  32. 2024Peering into the Mind of Language Models: An Approach for Attribution in Contextual Question Answeringcited by 25
  33. 2024A Survey on the Honesty of Large Language Modelscited by 23
  34. 2023Towards Evaluating AI Systems for Moral Status Using Self-Reportscited by 22
  35. 2024Language Models in Dialogue: Conversational Maxims for Human-AI Interactionscited by 21
  36. 2024HonestLLM: Toward an Honest and Helpful Large Language Modelcited by 18
  37. 2025Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Taskscited by 18
  38. 2024Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Modelscited by 15

50 most cited of 113 All 113 (JSON)