# Language Models Hallucinate, but May Excel at Fact Verification

arXiv: https://arxiv.org/abs/2310.14564 · HTML: https://mlsanity.com/paper/2310.14564/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 83 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 30 of 50)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [PAWS: Paraphrase Adversaries from Word Scrambling](https://mlsanity.com/paper/1904.01130/) (2019)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)

## Built on by (most cited first, 30 of 37)

- [MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents](https://mlsanity.com/paper/2404.10774/) (2024)
- [Language Models with Conformal Factuality Guarantees](https://mlsanity.com/paper/2402.10978/) (2024)
- [VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation](https://mlsanity.com/paper/2406.19276/) (2024)
- [LLM-based NLG Evaluation: Current Status and Challenges](https://mlsanity.com/paper/2402.01383/) (2024)
- [Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attacks](https://mlsanity.com/paper/2409.08087/) (2024)
- [AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedback](https://mlsanity.com/paper/2402.01469/) (2024)
- [Claim Verification in the Age of Large Language Models: A Survey](https://mlsanity.com/paper/2408.14317/) (2024)
- [Generative Large Language Models in Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2407.02351/) (2024)
- [Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verification](https://mlsanity.com/paper/2402.03686/) (2024)
- [ClaimVer: Explainable Claim-Level Verification and Evidence Attribution of Text Through Knowledge Graphs](https://mlsanity.com/paper/2403.09724/) (2024)
- [REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy](https://mlsanity.com/paper/2406.07735/) (2024)
- [DatBench: Discriminative, Faithful, and Efficient VLM Evaluations](https://mlsanity.com/paper/2601.02316/) (2026)
- [Truth over Tricks: Measuring and Mitigating Shortcut Learning in Misinformation Detection](https://mlsanity.com/paper/2506.02350/) (2025)
- [FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMs](https://mlsanity.com/paper/2510.12839/) (2025)
- [CMIE: Combining MLLM Insights with External Evidence for Explainable Out-of-Context Misinformation Detection](https://mlsanity.com/paper/2505.23449/) (2025)
- [Dynamic-KGQA: A Scalable Framework for Generating Adaptive Question Answering Datasets](https://mlsanity.com/paper/2503.05049/) (2025)
- [Veri-R1: Toward Precise and Faithful Claim Verification via Online Reinforcement Learning](https://mlsanity.com/paper/2510.01932/) (2025)
- [A Graph-based Verification Framework for Fact-Checking](https://mlsanity.com/paper/2503.07282/) (2025)
- [The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality](https://mlsanity.com/paper/2507.08371/) (2025)
- [How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models](https://mlsanity.com/paper/2407.00369/) (2024)
- [CrediRAG: Network-Augmented Credibility-Based Retrieval for Misinformation Detection in Reddit](https://mlsanity.com/paper/2410.12061/) (2024)
- [Speaking at the Right Level: Literacy-Controlled Counterspeech Generation with RAG-RL](https://mlsanity.com/paper/2509.01058/) (2025)
- [Exposing Citation Vulnerabilities in Generative Engines](https://mlsanity.com/paper/2510.06823/) (2025)
- [Discrepancy Detection at the Data Level: Toward Consistent Multilingual Question Answering](https://mlsanity.com/paper/2510.11928/) (2025)
- [Recall Them All: Retrieval-Augmented Language Models for Long Object List Extraction from Long Documents](https://mlsanity.com/paper/2405.02732/) (2024)
- [LEAF: Learning and Evaluation Augmented by Fact-Checking to Improve Factualness in Large Language Models](https://mlsanity.com/paper/2410.23526/) (2024)
- [FarExStance: Explainable Stance Detection for Farsi](https://mlsanity.com/paper/2412.14008/) (2024)
- [Self-Rationalization in the Wild: A Large Scale Out-of-Distribution Evaluation on NLI-related tasks](https://mlsanity.com/paper/2502.04797/) (2025)
- [CRAVE: A Conflicting Reasoning Approach for Explainable Claim Verification Using LLMs](https://mlsanity.com/paper/2504.14905/) (2025)
- [Inter-Passage Verification for Multi-evidence Multi-answer QA](https://mlsanity.com/paper/2506.00425/) (2025)

## Abstract (arXiv)

Recent progress in natural language processing (NLP) owes much to remarkable advances in large language models (LLMs). Nevertheless, LLMs frequently "hallucinate," resulting in non-factual outputs. Our carefully-designed human evaluation substantiates the serious hallucination issue, revealing that even GPT-3.5 produces factual outputs less than 25% of the time. This underscores the importance of fact verifiers in order to measure and incentivize progress. Our systematic investigation affirms that LLMs can be repurposed as effective fact verifiers with strong correlations with human judgments. Surprisingly, FLAN-T5-11B, the least factual generator in our study, performs the best as a fact verifier, even outperforming more capable LLMs like GPT3.5 and ChatGPT. Delving deeper, we analyze the reliance of these LLMs on high-quality evidence, as well as their deficiencies in robustness and generalization ability. Our study presents insights for developing trustworthy generation models.