VeriTrail: Closed-Domain Hallucination Detection with Traceability
Not on the map. Why?
Abstract (arXiv)
Even when instructed to adhere to source material, language models often generate unsubstantiated content - a phenomenon known as "closed-domain hallucination." This risk is amplified in processes with multiple generative steps (MGS), compared to processes with a single generative step (SGS). However, due to the greater complexity of MGS processes, we argue that detecting hallucinations in their final outputs is necessary but not sufficient: it is equally important to trace where hallucinated content was likely introduced and how faithful content may have been derived from the source material through intermediate outputs. To address this need, we present VeriTrail, the first closed-domain hallucination detection method designed to provide traceability for both MGS and SGS processes. We also introduce the first datasets to include all intermediate outputs as well as human annotations of final outputs' faithfulness for their respective MGS processes. We demonstrate that VeriTrail outperforms baseline methods on both datasets.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 27 May 2025.
Builds on
43 of 58 references.
References · 43
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
- 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
- 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
- 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
31 more
- 2018From Louvain to Leiden: guaranteeing well-connected communitiescited by 379
- 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
- 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
- 2021Recursively Summarizing Books with Human Feedbackcited by 287
- 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
- 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
- 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
- 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 241
- 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
- 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
- 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
- 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 186
- 2020What Can We Learn from Collective Human Opinions on Natural Language Inference Data?cited by 148
- 2024Long-form factuality in large language modelscited by 146
- 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
- 2023BooookScore: A systematic exploration of book-length summarization in the era of LLMscited by 137
- 2023Complex Claim Verification with Evidence Retrieved in the Wildcited by 110
- 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoningcited by 108
- 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretabilitycited by 100
- 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Modelscited by 70
- 2024FABLES: Evaluating faithfulness and content selection in book-length summarizationcited by 65
- 2023Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articlescited by 45
- 2021AmbiFC: Fact-Checking Ambiguous Claims with Evidencecited by 38
- 2024Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?cited by 34
- 2024Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Taskscited by 23
- 2025Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systemscited by 22
- 2025Towards Effective Extraction and Evaluation of Factual Claimscited by 18
- 2024The HalluRAG Dataset: Detecting Closed-Domain Hallucinations in RAG Applications Using an LLM's Internal Statescited by 8
- 2024Enhancing Incremental Summarization with Structured Representationscited by 4
- 2025MODS: Moderating a Mixture of Document Speakers to Summarize Debatable Queries in Document Collectionscited by 4
- 2024PRISM: Efficient Long-Range Reasoning With Short-Context LLMscited by 3
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.