# Retrieving Supporting Evidence for LLMs Generated Answers

arXiv: https://arxiv.org/abs/2306.13781 · HTML: https://mlsanity.com/paper/2306.13781/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2023.

## Built on by (most cited first, 21 of 21)

- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2311.09476/) (2023)
- [Cognitive Mirage: A Review of Hallucinations in Large Language Models](https://mlsanity.com/paper/2309.06794/) (2023)
- [RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledge](https://mlsanity.com/paper/2311.08147/) (2023)
- [Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models](https://mlsanity.com/paper/2407.06089/) (2024)
- [Hallucination Detection in Large Language Models with Metamorphic Relations](https://mlsanity.com/paper/2502.15844/) (2025)
- [A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagation](https://mlsanity.com/paper/2406.06950/) (2024)
- [Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Models](https://mlsanity.com/paper/2409.02976/) (2024)
- [HalluEntity: Benchmarking and Understanding Entity-Level Hallucination Detection](https://mlsanity.com/paper/2502.11948/) (2025)
- [FG-PRM: Fine-grained Hallucination Detection and Mitigation in Language Model Mathematical Reasoning](https://mlsanity.com/paper/2410.06304/) (2024)
- [AutoHall: Automated Factuality Hallucination Dataset Generation for Large Language Models](https://mlsanity.com/paper/2310.00259/) (2023)
- [QuIM-RAG: Advancing Retrieval-Augmented Generation with Inverted Question Matching for Enhanced QA Performance](https://mlsanity.com/paper/2501.02702/) (2025)
- [Post-Hoc Answer Attribution for Grounded and Trustworthy Long Document Comprehension: Task, Insights, and Challenges](https://mlsanity.com/paper/2406.06938/) (2024)
- [Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Models](https://mlsanity.com/paper/2510.06107/) (2025)
- [HIDE and Seek: Detecting Hallucinations in Language Models via Decoupled Representations](https://mlsanity.com/paper/2506.17748/) (2025)
- [Principled Detection of Hallucinations in Large Language Models via Multiple Testing](https://mlsanity.com/paper/2508.18473/) (2025)
- [Sequencing Matters: A Generate-Retrieve-Generate Model for Building Conversational Agents](https://mlsanity.com/paper/2311.09513/) (2023)
- [Unlocking Multi-View Insights in Knowledge-Dense Retrieval-Augmented Generation](https://mlsanity.com/paper/2404.12879/) (2024)
- [HealthBranches: Synthesizing Clinically-Grounded Question Answering Datasets via Decision Pathways](https://mlsanity.com/paper/2508.07308/) (2025)
- [Diagnosing Medical Datasets with Training Dynamics](https://mlsanity.com/paper/2411.01653/) (2024)
- [HalluDetect: Detecting, Mitigating, and Benchmarking Hallucinations in Conversational Systems in the Legal Domain](https://mlsanity.com/paper/2509.11619/) (2025)

## Abstract (arXiv)

Current large language models (LLMs) can exhibit near-human levels of performance on many natural language tasks, including open-domain question answering. Unfortunately, they also convincingly hallucinate incorrect answers, so that responses to questions must be verified against external sources before they can be accepted at face value. In this paper, we report a simple experiment to automatically verify generated answers against a corpus. After presenting a question to an LLM and receiving a generated answer, we query the corpus with the combination of the question + generated answer. We then present the LLM with the combination of the question + generated answer + retrieved answer, prompting it to indicate if the generated answer can be supported by the retrieved answer. We base our experiment on questions and passages from the MS MARCO (V1) test collection, exploring three retrieval approaches ranging from standard BM25 to a full question answering stack, including a reader based on the LLM. For a large fraction of questions, we find that an LLM is capable of verifying its generated answer if appropriate supporting material is provided. However, with an accuracy of 70-80%, this approach cannot be fully relied upon to detect hallucinations.