mlsanityData to 7 Oct 2026

On Hallucination and Predictive Uncertainty in Conditional Language Generation

arXiv 2103.15025 ↗First posted 28 Mar 2021cs.CLcited by 176

Not on the map. Why?

Abstract (arXiv)

Despite improvements in performances on different natural language generation tasks, deep neural models are prone to hallucinating facts that are incorrect or nonexistent. Different hypotheses are proposed and examined separately for different tasks, but no systematic explanations are available across these tasks. In this study, we draw connections between hallucinations and predictive uncertainty in conditional language generation. We investigate their relationship in both image captioning and data-to-text generation and propose a simple extension to beam search to reduce hallucination. Our analysis shows that higher predictive uncertainty corresponds to a higher chance of hallucination. Epistemic uncertainty is more indicative of hallucination than aleatoric or total uncertainties. It helps to achieve better results of trading performance in standard metric for less hallucination with the proposed beam search variant.

Key facts. Cited by 176 papers in the map (data through 7 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 28 Mar 2021.

Builds on

25 of 35 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  7. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  9. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  10. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  11. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  12. 2018Object Hallucination in Image Captioningcited by 661
13 more
  1. 2015Weight Uncertainty in Neural Networkscited by 554
  2. 2016Self-critical Sequence Training for Image Captioningcited by 544
  3. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
  4. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 267
  5. 2017Challenges in Data-to-Document Generationcited by 226
  6. 2017Faithful to the Original: Fact Aware Neural Abstractive Summarizationcited by 141
  7. 2018Analyzing Uncertainty in Neural Machine Translationcited by 126
  8. 2018Quantifying Uncertainties in Natural Language Processing Taskscited by 79
  9. 2018Discriminability objective for training descriptive captionscited by 67
  10. 2019Domain Robustness in Neural Machine Translationcited by 64
  11. 2019Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generationcited by 62
  12. 2017Deep and Confident Prediction for Time Series at Ubercited by 50
  13. 2020Understanding Neural Abstractive Summarization Models via Uncertaintycited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 14; 2023: 44; 2024: 57; 2025: 43; 2026: 16.

176 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  4. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  5. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  6. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  7. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
  8. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  9. 2023Challenges and Applications of Large Language Modelscited by 251
  10. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  11. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 155
  12. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 130
38 more
  1. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  2. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  3. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 98
  4. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 89
  5. 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Modelscited by 80
  6. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 80
  7. 2025SAFE: Multitask Failure Detection for Vision-Language-Action Modelscited by 74
  8. 2023On the Calibration of Large Language Models and Alignmentcited by 72
  9. 2022Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-trainingcited by 71
  10. 2023Automatic Evaluation of Attribution by Large Language Modelscited by 70
  11. 2025Evaluating Step-by-step Reasoning Traces: A Surveycited by 62
  12. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  13. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewritescited by 61
  14. 2023Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Modelscited by 60
  15. 2024A Survey of Uncertainty Estimation in LLMs: Theory Meets Practicecited by 54
  16. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  17. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  18. 2024Uncertainty Quantification for In-Context Learning of Large Language Modelscited by 45
  19. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  20. 2024Multicalibration for Confidence Scoring in LLMscited by 44
  21. 2023EHRSQL: A Practical Text-to-SQL Benchmark for Electronic Health Recordscited by 42
  22. 2022SafeText: A Benchmark for Exploring Physical Safety in Language Modelscited by 41
  23. 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Modelscited by 40
  24. 2024Uncertainty in Language Models: Assessment through Rank-Calibrationcited by 40
  25. 2024Detecting Hallucinations in Large Language Model Generation: A Token Probability Approachcited by 40
  26. 2024Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinationscited by 38
  27. 2021Factual Consistency Evaluation for Text Summarization via Counterfactual Estimationcited by 37
  28. 2025Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?cited by 35
  29. 2024AI Hallucinations: A Misnomer Worth Clarifyingcited by 34
  30. 2024Improving Uncertainty Estimation through Semantically Diverse Language Generationcited by 34
  31. 2023Can Large Language Models Capture Dissenting Human Voices?cited by 33
  32. 2025MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Modelscited by 33
  33. 2024Towards Uncertainty-Aware Language Agentcited by 29
  34. 2022Learning to Revise References for Faithful Summarizationcited by 28
  35. 2023Tailoring Language Generation Models under Total Variation Distancecited by 27
  36. 2024Cycles of Thought: Measuring LLM Confidence through Stable Explanationscited by 27
  37. 2023On Diversified Preferences of Large Language Model Alignmentcited by 26
  38. 2024Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trendscited by 26

50 most cited of 176 All 176 (JSON)