mlsanityData to 7 Oct 2026

Retrieval Augmentation Reduces Hallucination in Conversation

arXiv 2104.07567 ↗First posted 15 Apr 2021cs.CL cs.AIcited by 691

Not on the map. Why?

Cited by 218 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Despite showing increasingly human-like conversational abilities, state-of-the-art dialogue models often suffer from factual incorrectness and hallucination of knowledge (Roller et al., 2020). In this work we explore the use of neural-retrieval-in-the-loop architectures - recently shown to be effective in open-domain QA (Lewis et al., 2020b; Izacard and Grave, 2020) - for knowledge-grounded dialogue, a task that is arguably more challenging as it requires querying based on complex multi-turn dialogue context and generating conversationally coherent responses. We study various types of architectures with multiple components - retrievers, rankers, and encoder-decoders - with the goal of maximizing knowledgeability while retaining conversational ability. We demonstrate that our best models obtain state-of-the-art performance on two knowledge-grounded conversational tasks. The models exhibit open-domain conversational capabilities, generalize effectively to scenarios not within the training data, and, as verified by human evaluations, substantially reduce the well-known problem of knowledge hallucination in state-of-the-art chatbots.

Key facts. Cited by 691 papers in the map (data through 7 Oct 2026). Builds on 52 of its 67 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

52 of 67 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2016Pointer Sentinel Mixture Modelscited by 2,992
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  11. 2017Billion-scale similarity search with GPUscited by 2,477
  12. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
38 more
  1. 2019Language Models as Knowledge Bases?cited by 1,922
  2. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  3. 2018Hierarchical Neural Story Generationcited by 1,182
  4. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  5. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  6. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  7. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  8. 2020Recipes for building an open-domain chatbotcited by 593
  9. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  10. 2019Defending Against Neural Fake Newscited by 553
  11. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  12. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  13. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  14. 2014Memory Networkscited by 354
  15. 2020The Pushshift Reddit Datasetcited by 289
  16. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  17. 2020Nearest Neighbor Machine Translationcited by 209
  18. 2020USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generationcited by 186
  19. 2019PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Textcited by 182
  20. 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 178
  21. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 164
  22. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 160
  23. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 151
  24. 2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
  25. 2017ParlAI: A Dialog Research Software Platformcited by 139
  26. 2017A Knowledge-Grounded Neural Conversation Modelcited by 136
  27. 2020Knowledge-Grounded Dialogue Generation with Pre-trained Language Modelscited by 107
  28. 2018A Dataset for Document Grounded Conversationscited by 104
  29. 2021Adaptive Semiparametric Language Modelscited by 83
  30. 2020Sequential Latent Knowledge Selection for Knowledge-Grounded Dialoguecited by 82
  31. 2016Improving Neural Language Models with a Continuous Cachecited by 81
  32. 2020Relevance-guided Supervision for OpenQA with ColBERTcited by 71
  33. 2019Knowledge Guided Text Retrieval and Reading for Open Domain Question Answeringcited by 59
  34. 2020Low-Resource Knowledge-Grounded Dialogue Generationcited by 53
  35. 2020Augmenting Transformers with KNN-Based Composite Memory for Dialoguecited by 53
  36. 2019Improving Question Answering over Incomplete KBs with Knowledge-Aware Readercited by 37
  37. 2021Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialogcited by 33
  38. 2020A Corpus of Controlled Opinionated and Knowledgeable Movie Discussions for Training Neural Conversation Modelscited by 8

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 27; 2022: 62; 2023: 127; 2024: 206; 2025: 161; 2026: 108.

691 papers.

Sort:

By area:

  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  5. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  6. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  7. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  9. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  10. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  11. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  12. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
38 more
  1. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  2. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  3. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  4. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  5. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  6. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  7. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  8. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  9. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  10. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  11. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  12. 2023How Language Model Hallucinations Can Snowballcited by 310
  13. 2023Tool Learning with Foundation Modelscited by 302
  14. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  15. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
  16. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
  17. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  18. 2023Challenges and Applications of Large Language Modelscited by 251
  19. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaborationcited by 246
  20. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  21. 2021Internet-Augmented Dialogue Generationcited by 217
  22. 2025Why Language Models Hallucinatecited by 205
  23. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 203
  24. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  25. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  26. 2024Corrective Retrieval Augmented Generationcited by 187
  27. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 167
  28. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
  29. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)cited by 159
  30. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
  31. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  32. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  33. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  34. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 144
  35. 2023PaperQA: Retrieval-Augmented Generative Agent for Scientific Researchcited by 143
  36. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 137
  37. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  38. 2022Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answeringcited by 126

50 most cited of 691 All 691 (JSON)