# DelucionQA: Detecting Hallucinations in Domain-specific Question Answering

arXiv: https://arxiv.org/abs/2312.05200 · HTML: https://mlsanity.com/paper/2312.05200/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 8 Dec 2023.

## Builds on (most cited first, 14 of 14)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics](https://mlsanity.com/paper/2104.13346/) (2021)
- [Measuring Attribution in Natural Language Generation Models](https://mlsanity.com/paper/2112.12870/) (2021)
- [Detecting Hallucinated Content in Conditional Neural Sequence Generation](https://mlsanity.com/paper/2011.02593/) (2020)
- [A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation](https://mlsanity.com/paper/2104.08704/) (2021)
- [Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation](https://mlsanity.com/paper/2105.08504/) (2021)
- [Read before Generate! Faithful Long Form Question Answering with Machine Reading](https://mlsanity.com/paper/2203.00343/) (2022)

## Built on by (most cited first, 30 of 43)

- [RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2407.11005/) (2024)
- [HalluLens: LLM Hallucination Benchmark](https://mlsanity.com/paper/2504.17550/) (2025)
- [A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations](https://mlsanity.com/paper/2407.04069/) (2024)
- [Alleviating Hallucinations of Large Language Models through Induced Hallucinations](https://mlsanity.com/paper/2312.15710/) (2023)
- [LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions](https://mlsanity.com/paper/2509.18970/) (2025)
- [Is Semantic Chunking Worth the Computational Cost?](https://mlsanity.com/paper/2410.13070/) (2024)
- [GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation](https://mlsanity.com/paper/2502.01113/) (2025)
- [CARROT: A Cost Aware Rate Optimal Router](https://mlsanity.com/paper/2502.03261/) (2025)
- [CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging](https://mlsanity.com/paper/2502.05664/) (2025)
- [Mitigating Large Language Model Hallucination with Faithful Finetuning](https://mlsanity.com/paper/2406.11267/) (2024)
- [Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settings](https://mlsanity.com/paper/2503.15620/) (2025)
- [ProVision: Programmatically Scaling Vision-centric Instruction Data for Multimodal Language Models](https://mlsanity.com/paper/2412.07012/) (2024)
- [The Order Effect: Investigating Prompt Sensitivity to Input Order in LLMs](https://mlsanity.com/paper/2502.04134/) (2025)
- [An Audit on the Perspectives and Challenges of Hallucinations in NLP](https://mlsanity.com/paper/2404.07461/) (2024)
- [Optimal Query Allocation in Extractive QA with LLMs: A Learning-to-Defer Framework with Theoretical Guarantees](https://mlsanity.com/paper/2410.15761/) (2024)
- [AuditLLM: A Tool for Auditing Large Language Models Using Multiprobe Approach](https://mlsanity.com/paper/2402.09334/) (2024)
- [GenAudit: Fixing Factual Errors in Language Model Outputs with Evidence](https://mlsanity.com/paper/2402.12566/) (2024)
- [Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversations](https://mlsanity.com/paper/2402.11770/) (2024)
- [ERBench: An Entity-Relationship based Automatically Verifiable Hallucination Benchmark for Large Language Models](https://mlsanity.com/paper/2403.05266/) (2024)
- [Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modeling](https://mlsanity.com/paper/2409.11283/) (2024)
- [EnronQA: Towards Personalized RAG over Private Documents](https://mlsanity.com/paper/2505.00263/) (2025)
- [LLMAuditor: A Framework for Auditing Large Language Models Using Human-in-the-Loop](https://mlsanity.com/paper/2402.09346/) (2024)
- [Evaluating Consistency and Reasoning Capabilities of Large Language Models](https://mlsanity.com/paper/2404.16478/) (2024)
- [Corpus2Skill: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG](https://mlsanity.com/paper/2604.14572/) (2026)
- [HypoTermQA: Hypothetical Terms Dataset for Benchmarking Hallucination Tendency of LLMs](https://mlsanity.com/paper/2402.16211/) (2024)
- [Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directions](https://mlsanity.com/paper/2409.19747/) (2024)
- [Analysis of Plan-based Retrieval for Grounded Text Generation](https://mlsanity.com/paper/2408.10490/) (2024)
- [CLUE: Concept-Level Uncertainty Estimation for Large Language Models](https://mlsanity.com/paper/2409.03021/) (2024)
- [LightHouse: A Survey of AGI Hallucination](https://mlsanity.com/paper/2401.06792/) (2024)
- [THaMES: An End-to-End Tool for Hallucination Mitigation and Evaluation in Large Language Models](https://mlsanity.com/paper/2409.11353/) (2024)

## Abstract (arXiv)

Hallucination is a well-known phenomenon in text generated by large language models (LLMs). The existence of hallucinatory responses is found in almost all application scenarios e.g., summarization, question-answering (QA) etc. For applications requiring high reliability (e.g., customer-facing assistants), the potential existence of hallucination in LLM-generated text is a critical problem. The amount of hallucination can be reduced by leveraging information retrieval to provide relevant background information to the LLM. However, LLMs can still generate hallucinatory content for various reasons (e.g., prioritizing its parametric knowledge over the context, failure to capture the relevant information from the context, etc.). Detecting hallucinations through automated methods is thus paramount. To facilitate research in this direction, we introduce a sophisticated dataset, DelucionQA, that captures hallucinations made by retrieval-augmented LLMs for a domain-specific QA task. Furthermore, we propose a set of hallucination detection methods to serve as baselines for future works from the research community. Analysis and case study are also provided to share valuable insights on hallucination phenomena in the target scenario.