mlsanityData to 7 Oct 2026

On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?

arXiv 2204.07931 ↗First posted 17 Apr 2022cs.CLcited by 137

Not on the map. Why?

Abstract (arXiv)

Knowledge-grounded conversational models are known to suffer from producing factually invalid statements, a phenomenon commonly called hallucination. In this work, we investigate the underlying causes of this phenomenon: is hallucination due to the training data, or to the models? We conduct a comprehensive human study on both existing knowledge-grounded conversational benchmarks and several state-of-the-art models. Our study reveals that the standard benchmarks consist of >60% hallucinated responses, leading to models that not only hallucinate but even amplify hallucinations. Our findings raise important questions on the quality of existing datasets and models trained using them. We make our annotations publicly available for future research.

Key facts. Cited by 137 papers in the map (data through 7 Oct 2026). Builds on 21 of its 36 references that are in the map. First posted on arXiv 17 Apr 2022.

Builds on

21 of 36 references.

References · 21

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  7. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  8. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  9. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  10. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  11. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
  12. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
9 more
  1. 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
  2. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  3. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  4. 2021Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Groundingcited by 106
  5. 2018A Dataset for Document Grounded Conversationscited by 104
  6. 2020Improved Natural Language Generation via Loss Truncationcited by 75
  7. 2021Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Featurescited by 74
  8. 2021Focused Attention Improves Document-Grounded Generationcited by 24
  9. 2021A Large-Scale Analysis of Mixed Initiative in Information-Seeking Dialogues for Conversational Searchcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 52; 2024: 46; 2025: 21; 2026: 8.

137 papers.

Sort:

By area:

  1. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  2. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  3. 2023Explainability for Large Language Models: A Surveycited by 331
  4. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  5. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 223
  6. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  7. 2023Simulating Opinion Dynamics with Networks of LLM-based Agentscited by 153
  8. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  9. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 146
  10. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 130
  11. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 118
  12. 2023Calibrated Language Models Must Hallucinatecited by 114
38 more
  1. 2023Evaluating Open-Domain Question Answering in the Era of Large Language Modelscited by 110
  2. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  3. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 106
  4. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  5. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 102
  6. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 98
  7. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contextscited by 92
  8. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  9. 2023Exposing Attention Glitches with Flip-Flop Language Modelingcited by 74
  10. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignmentcited by 56
  11. 2023HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attributioncited by 55
  12. 2024Lynx: An Open Source Hallucination Evaluation Modelcited by 55
  13. 2024Attribute First, then Generate: Locally-attributable Grounded Text Generationcited by 54
  14. 2023SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Modelscited by 53
  15. 2023DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agentscited by 52
  16. 2022RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Groundingcited by 47
  17. 2024Confabulation: The Surprising Value of Large Language Model Hallucinationscited by 42
  18. 2024AI Hallucinations: A Misnomer Worth Clarifyingcited by 34
  19. 2023Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuningcited by 33
  20. 2023Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AIcited by 33
  21. 2023Diving Deep into Modes of Fact Hallucinations in Dialogue Systemscited by 32
  22. 2023Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AIcited by 32
  23. 2023Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledgecited by 32
  24. 2024SAC-KG: Exploiting Large Language Models as Skilled Automatic Constructors for Domain Knowledge Graphscited by 31
  25. 2023Elastic Weight Removal for Faithful and Abstractive Dialogue Generationcited by 29
  26. 2024HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wildcited by 27
  27. 2023The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answeringcited by 26
  28. 2024AttributionBench: How Hard is Automatic Attribution Evaluation?cited by 26
  29. 2023It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performancecited by 25
  30. 2023Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrievalcited by 24
  31. 2024Mitigating Large Language Model Hallucination with Faithful Finetuningcited by 24
  32. 2024Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Modelscited by 24
  33. 2023Scientists' Perspectives on the Potential for Generative AI in their Fieldscited by 22
  34. 2023KILM: Knowledge Injection into Encoder-Decoder Language Modelscited by 19
  35. 2023Koala: An Index for Quantifying Overlaps with Pre-training Corporacited by 19
  36. 2025Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Modelscited by 18
  37. 2023Leftover Lunch: Advantage-based Offline Reinforcement Learning for Language Modelscited by 17
  38. 2023TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documentscited by 17

50 most cited of 137 All 137 (JSON)