mlsanityData to 7 Oct 2026

DelucionQA: Detecting Hallucinations in Domain-specific Question Answering

arXiv 2312.05200 ↗First posted 8 Dec 2023cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

Hallucination is a well-known phenomenon in text generated by large language models (LLMs). The existence of hallucinatory responses is found in almost all application scenarios e.g., summarization, question-answering (QA) etc. For applications requiring high reliability (e.g., customer-facing assistants), the potential existence of hallucination in LLM-generated text is a critical problem. The amount of hallucination can be reduced by leveraging information retrieval to provide relevant background information to the LLM. However, LLMs can still generate hallucinatory content for various reasons (e.g., prioritizing its parametric knowledge over the context, failure to capture the relevant information from the context, etc.). Detecting hallucinations through automated methods is thus paramount. To facilitate research in this direction, we introduce a sophisticated dataset, DelucionQA, that captures hallucinations made by retrieval-augmented LLMs for a domain-specific QA task. Furthermore, we propose a set of hallucination detection methods to serve as baselines for future works from the research community. Analysis and case study are also provided to share valuable insights on hallucination phenomena in the target scenario.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 14 of its 26 references that are in the map. First posted on arXiv 8 Dec 2023.

Builds on

14 of 26 references.

References · 14

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2017Billion-scale similarity search with GPUscited by 2,477
  4. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  5. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  6. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  7. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  8. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  9. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 285
  10. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  11. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 151
  12. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 137
2 more
  1. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  2. 2022Read before Generate! Faithful Long Form Question Answering with Machine Readingcited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 22; 2025: 16; 2026: 4.

43 papers.

Sort:

By area:

  1. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 115
  2. 2025HalluLens: LLM Hallucination Benchmarkcited by 88
  3. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
  4. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 78
  5. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  6. 2024Is Semantic Chunking Worth the Computational Cost?cited by 43
  7. 2025GFM-RAG: Graph Foundation Model for Retrieval Augmented Generationcited by 40
  8. 2025CARROT: A Cost Aware Rate Optimal Routercited by 34
  9. 2025CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debuggingcited by 31
  10. 2024Mitigating Large Language Model Hallucination with Faithful Finetuningcited by 24
  11. 2025Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settingscited by 22
  12. 2024ProVision: Programmatically Scaling Vision-centric Instruction Data for Multimodal Language Modelscited by 21
31 more
  1. 2025The Order Effect: Investigating Prompt Sensitivity to Input Order in LLMscited by 19
  2. 2024An Audit on the Perspectives and Challenges of Hallucinations in NLPcited by 17
  3. 2024Optimal Query Allocation in Extractive QA with LLMs: A Learning-to-Defer Framework with Theoretical Guaranteescited by 15
  4. 2024AuditLLM: A Tool for Auditing Large Language Models Using Multiprobe Approachcited by 11
  5. 2024GenAudit: Fixing Factual Errors in Language Model Outputs with Evidencecited by 10
  6. 2024Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversationscited by 9
  7. 2024ERBench: An Entity-Relationship based Automatically Verifiable Hallucination Benchmark for Large Language Modelscited by 9
  8. 2024Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modelingcited by 9
  9. 2024LLMAuditor: A Framework for Auditing Large Language Models Using Human-in-the-Loopcited by 8
  10. 2025EnronQA: Towards Personalized RAG over Private Documentscited by 8
  11. 2024Evaluating Consistency and Reasoning Capabilities of Large Language Modelscited by 7
  12. 2024HypoTermQA: Hypothetical Terms Dataset for Benchmarking Hallucination Tendency of LLMscited by 6
  13. 2026Corpus2Skill: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAGcited by 6
  14. 2024Analysis of Plan-based Retrieval for Grounded Text Generationcited by 5
  15. 2024Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directionscited by 5
  16. 2024LightHouse: A Survey of AGI Hallucinationcited by 4
  17. 2024CLUE: Concept-Level Uncertainty Estimation for Large Language Modelscited by 4
  18. 2024THaMES: An End-to-End Tool for Hallucination Mitigation and Evaluation in Large Language Modelscited by 4
  19. 2025CLEAR: Error Analysis via LLM-as-a-Judge Made Easycited by 3
  20. 2024Genetic Approach to Mitigate Hallucination in Generative IRcited by 2
  21. 2025The Art of Saying "Maybe": A Conformal Lens for Uncertainty Benchmarking in VLMscited by 2
  22. 2026Retrieving Minimal and Sufficient Reasoning Subgraphs with Graph Foundation Models for Path-aware GraphRAGcited by 2
  23. 2025OAEI-LLM-T: A TBox Benchmark Dataset for Understanding Large Language Model Hallucinations in Ontology Matchingcited by 1
  24. 2025PrismRAG: Boosting RAG Factuality with Distractor Resilience and Strategized Reasoningcited by 1
  25. 2025From Search to Reasoning: A Five-Level RAG Capability Framework for Enterprise Datacited by 1
  26. 2025The Gray Zone of Faithfulness: Taming Ambiguity in Unfaithfulness Detectioncited by 1
  27. 2024Detecting Response Generation Not Requiring Factual Judgmentcited by 0
  28. 2025Conservative Bias in Large Language Models: Measuring Relation Predictionscited by 0
  29. 2025A Context-Aware Dual-Metric Framework for Confidence Estimation in Large Language Modelscited by 0
  30. 2026GLM-RAG: Graph Language Models for Graph-Based Retrieval-Augmented Generationcited by 0
  31. 2026Evidence-Aligned Entity Verification for Hallucination Detection in Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
  2. 2024Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMs
  3. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
  4. 2025REFIND at SemEval-2025 Task 3: Retrieval-Augmented Factuality Hallucination Detection in Large Language Models
  5. 2025SelfCheck-Eval: A Multi-Module Framework for Zero-Resource Hallucination Detection in Large Language Models
  6. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models
  7. 2024FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs
  8. 2026Evaluating Hallucinations in Domain-Adapted Large Language Models