mlsanityData to 7 Oct 2026

FaithDial: A Faithful Benchmark for Information-Seeking Dialogue

arXiv 2204.10757 ↗First posted 22 Apr 2022cs.CLcited by 108

Not on the map. Why?

Abstract (arXiv)

The goal of information-seeking dialogue is to respond to seeker queries with natural language utterances that are grounded on knowledge sources. However, dialogue systems often produce unsupported utterances, a phenomenon known as hallucination. To mitigate this behavior, we adopt a data-centric solution and create FaithDial, a new benchmark for hallucination-free dialogues, by editing hallucinated responses in the Wizard of Wikipedia (WoW) benchmark. We observe that FaithDial is more faithful than WoW while also maintaining engaging conversations. We show that FaithDial can serve as training signal for: i) a hallucination critic, which discriminates whether an utterance is faithful or not, and boosts the performance by 12.8 F1 score on the BEGIN benchmark compared to existing datasets for dialogue coherence; ii) high-quality dialogue generation. We benchmark a series of state-of-the-art models and propose an auxiliary contrastive objective that achieves the highest level of faithfulness and abstractiveness based on several automated metrics. Further, we find that the benefits of FaithDial generalize to zero-shot transfer on other datasets, such as CMU-Dog and TopicalChat. Finally, human evaluation reveals that responses generated by models trained on FaithDial are perceived as more interpretable, cooperative, and engaging.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 41 of its 72 references that are in the map. First posted on arXiv 22 Apr 2022.

Builds on

41 of 72 references.

References · 41

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  9. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  10. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  11. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  12. 2020Recipes for building an open-domain chatbotcited by 593
29 more
  1. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  2. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  3. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 292
  4. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
  5. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 267
  6. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  7. 2017Challenges in Data-to-Document Generationcited by 226
  8. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 209
  9. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  10. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 137
  11. 2018Dialogue Natural Language Inferencecited by 135
  12. 2021Annotating and Modeling Fine-grained Factuality in Summarizationcited by 131
  13. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
  14. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  15. 2021CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarizationcited by 120
  16. 2020Knowledge-Grounded Dialogue Generation with Pre-trained Language Modelscited by 107
  17. 2021Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Groundingcited by 106
  18. 2018A Dataset for Document Grounded Conversationscited by 104
  19. 2020Improved Natural Language Generation via Loss Truncationcited by 75
  20. 2021Evaluating Attribution in Dialogue Systems: The BEGIN Benchmarkcited by 75
  21. 2021Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Featurescited by 74
  22. 2020I like fish, especially dolphins: Addressing Contradictions in Dialogue Modelingcited by 68
  23. 2021DialFact: A Benchmark for Fact-Checking in Dialoguecited by 62
  24. 2019Evaluating Coherence in Dialogue Systems using Entailmentcited by 53
  25. 2021Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarizationcited by 50
  26. 2021CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuningcited by 44
  27. 2021Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generationcited by 29
  28. 2021Focused Attention Improves Document-Grounded Generationcited by 24
  29. 2021On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependenciescited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 31; 2024: 27; 2025: 23; 2026: 23.

108 papers.

Sort:

By area:

  1. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  2. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  3. 2023MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problemscited by 128
  4. 2024Controllable Preference Optimization: Toward Controllable Multi-Objective Alignmentcited by 105
  5. 2023The Generative AI Paradox: "What It Can Create, It May Not Understand"cited by 97
  6. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  7. 2023Model Merging by Uncertainty-Based Gradient Matchingcited by 77
  8. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 72
  9. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 50
  10. 2023Detecting and Mitigating Hallucinations in Multilingual Summarisationcited by 48
  11. 2024VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulationcited by 43
  12. 2023Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generatorscited by 42
38 more
  1. 2024Confabulation: The Surprising Value of Large Language Model Hallucinationscited by 42
  2. 2023Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preferencecited by 37
  3. 2024AI Hallucinations: A Misnomer Worth Clarifyingcited by 34
  4. 2023Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuningcited by 33
  5. 2023Diving Deep into Modes of Fact Hallucinations in Dialogue Systemscited by 32
  6. 2023HallE-Control: Controlling Object Hallucination in Large Multimodal Modelscited by 31
  7. 2023Elastic Weight Removal for Faithful and Abstractive Dialogue Generationcited by 29
  8. 2023FactCHD: Benchmarking Fact-Conflicting Hallucination Detectioncited by 29
  9. 2023The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answeringcited by 26
  10. 2024HalluDial: A Large-Scale Benchmark for Automatic Dialogue-Level Hallucination Evaluationcited by 26
  11. 2024RefChecker: Reference-based Fine-grained Hallucination Checker and Benchmark for Large Language Modelscited by 24
  12. 2025The Illusion of Progress: Re-evaluating Hallucination Detection in LLMscited by 22
  13. 2023Zero-shot Faithfulness Evaluation for Text Summarization with Foundation Language Modelcited by 21
  14. 2024CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Modelscited by 18
  15. 2023Leftover Lunch: Advantage-based Offline Reinforcement Learning for Language Modelscited by 17
  16. 2024An Audit on the Perspectives and Challenges of Hallucinations in NLPcited by 17
  17. 2024DiaHalu: A Dialogue-level Hallucination Evaluation Benchmark for Large Language Modelscited by 15
  18. 2024Towards Rationality in Language and Multimodal Agents: A Surveycited by 15
  19. 2024Improving Model Factuality with Fine-grained Critique-based Evaluatorcited by 15
  20. 2023CHAMPAGNE: Learning Real-world Conversation from Large-Scale Web Videoscited by 14
  21. 2023Leveraging Large Language Models for Automated Dialogue Analysiscited by 14
  22. 2024ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Modelscited by 14
  23. 2024Fakes of Varying Shades: How Warning Affects Human Perception and Engagement Regarding LLM Hallucinationscited by 12
  24. 2023Pointwise Mutual Information Based Metric and Decoding Strategy for Faithful Generation in Document Grounded Dialogscited by 11
  25. 2025UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversationscited by 11
  26. 2023textitDial BeInfo for Faithfulness: Improving Factuality of Information-Seeking Dialogue via Behavioural Fine-Tuningcited by 9
  27. 2022Target-Guided Dialogue Response Generation Using Commonsense and Data Augmentationcited by 8
  28. 2022What Makes Data-to-Text Generation Hard for Pretrained Language Models?cited by 8
  29. 2024SudoLM: Learning Access Control of Parametric Knowledge with Authorization Alignmentcited by 8
  30. 2026MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversationscited by 8
  31. 2023Improved Instruction Ordering in Recipe-Grounded Conversationcited by 7
  32. 2024A Survey of Automatic Hallucination Evaluation on Natural Language Generationcited by 7
  33. 2024ANAH: Analytical Annotation of Hallucinations in Large Language Modelscited by 7
  34. 2023Insights into Classifying and Mitigating LLMs' Hallucinationscited by 6
  35. 2025Evaluating Evaluation Metrics -- The Mirage of Hallucination Detectioncited by 6
  36. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 6
  37. 2025Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generationcited by 6
  38. 2023Sequence-Level Certainty Reduces Hallucination In Knowledge-Grounded Dialogue Generationcited by 5

50 most cited of 108 All 108 (JSON)