mlsanityData to 14 Oct 2026

TruthfulQA: Measuring How Models Mimic Human Falsehoods

arXiv 2109.07958 ↗First posted 8 Sep 2021cs.CL cs.AI cs.CYcited by 2,975

Not on the map. Why?

Cited by 365 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics. We crafted questions that some humans would answer falsely due to a false belief or misconception. To perform well, models must avoid generating false answers learned from imitating human texts. We tested GPT-3, GPT-Neo/J, GPT-2 and a T5-based model. The best model was truthful on 58% of questions, while human performance was 94%. Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans. The largest models were generally the least truthful. This contrasts with other NLP tasks, where performance improves with model size. However, this result is expected if false answers are learned from the training distribution. We suggest that scaling up models alone is less promising for improving truthfulness than fine-tuning using training objectives other than imitation of text from the web.

Key facts. Cited by 2,975 papers in the map (data through 14 Oct 2026). Builds on 39 of its 64 references that are in the map. First posted on arXiv 8 Sep 2021.

Builds on

39 of 64 references.

References · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  11. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  12. 2020Learning to summarize from human feedbackcited by 2,588
27 more
  1. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  2. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  3. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  4. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  5. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  6. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  7. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  8. 2020Recipes for building an open-domain chatbotcited by 593
  9. 2019Defending Against Neural Fake Newscited by 559
  10. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
  11. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 378
  12. 2021True Few-Shot Learning with Language Modelscited by 344
  13. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  14. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 181
  15. 2020Recipes for Safety in Open-domain Chatbotscited by 170
  16. 2021Alignment of Language Agentscited by 168
  17. 2021Hurdles to Progress in Long-form Question Answeringcited by 167
  18. 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 167
  19. 2021Understanding the Capabilities, Limitations, and Societal Impact of Large Language Modelscited by 160
  20. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  21. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 152
  22. 2021Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challengecited by 80
  23. 2020The Radicalization Risks of GPT-3 and Advanced Neural Language Modelscited by 76
  24. 2020GO FIGURE: A Meta Evaluation of Factuality in Summarizationcited by 76
  25. 2018Predicting Factuality of Reporting and Bias of News Media Sourcescited by 71
  26. 2019Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modelingcited by 71
  27. 2018Trick Me If You Can: Human-in-the-loop Generation of Adversarial Examples for Question Answeringcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 50; 2023: 334; 2024: 816; 2025: 873; 2026: 895.

2,975 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  8. 2023A Survey of Large Language Modelscited by 2,576
  9. 2022Emergent Abilities of Large Language Modelscited by 2,512
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  11. 2024Qwen2 Technical Reportcited by 2,120
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
38 more
  1. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  2. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  4. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  6. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  7. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  8. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  10. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  11. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  12. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  13. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  14. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  15. 2022Red Teaming Language Models with Language Modelscited by 1,026
  16. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  17. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  18. 2025Large Language Diffusion Modelscited by 912
  19. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  20. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  21. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  22. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  23. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  24. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  25. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  26. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  27. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  28. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  29. 2022Galactica: A Large Language Model for Sciencecited by 744
  30. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  31. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
  32. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  33. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  34. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  35. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  36. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  37. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  38. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694

50 most cited of 2,975 All 2,975 (JSON)