mlsanityData to 7 Oct 2026

Language Models Hallucinate, but May Excel at Fact Verification

arXiv 2310.14564 ↗First posted 23 Oct 2023cs.CLcited by 35

Not on the map. Why?

Abstract (arXiv)

Recent progress in natural language processing (NLP) owes much to remarkable advances in large language models (LLMs). Nevertheless, LLMs frequently "hallucinate," resulting in non-factual outputs. Our carefully-designed human evaluation substantiates the serious hallucination issue, revealing that even GPT-3.5 produces factual outputs less than 25% of the time. This underscores the importance of fact verifiers in order to measure and incentivize progress. Our systematic investigation affirms that LLMs can be repurposed as effective fact verifiers with strong correlations with human judgments. Surprisingly, FLAN-T5-11B, the least factual generator in our study, performs the best as a fact verifier, even outperforming more capable LLMs like GPT3.5 and ChatGPT. Delving deeper, we analyze the reliance of these LLMs on high-quality evidence, as well as their deficiencies in robustness and generalization ability. Our study presents insights for developing trustworthy generation models.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 50 of its 83 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

50 of 83 references.

References · 50

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2017On Calibration of Modern Neural Networkscited by 4,860
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  12. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
38 more
  1. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  2. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  4. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  5. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  6. 2019How Can We Know What Language Models Know?cited by 953
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  8. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  9. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  10. 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
  11. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  12. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  13. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  14. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  15. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  16. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  17. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  18. 2021A Survey on Automated Fact-Checkingcited by 376
  19. 2017Fake News Detection on Social Media: A Data Mining Perspectivecited by 334
  20. 2023How Language Model Hallucinations Can Snowballcited by 310
  21. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  22. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  23. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  24. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 209
  25. 2020CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claimscited by 195
  26. 2023Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERTcited by 155
  27. 2017Automatic Detection of Fake Newscited by 145
  28. 2019Towards Debiasing Fact Verification Modelscited by 143
  29. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 137
  30. 2023Human-like Summarization Evaluation with ChatGPTcited by 126
  31. 2019Handling Divergent Reference Texts when Evaluating Table-to-Text Generationcited by 119
  32. 2021Entity-level Factual Consistency of Abstractive Text Summarizationcited by 118
  33. 2021Fool Me Twice: Entailment from Wikipedia Gamificationcited by 50
  34. 2020Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraintscited by 48
  35. 2021FaVIQ: FAct Verification from Information-seeking Questionscited by 36
  36. 2021InfoLM: A New Metric to Evaluate Summarization & Data2Text Generationcited by 33
  37. 2022CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generationcited by 29
  38. 2019Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 15; 2026: 5.

35 papers.

Sort:

By area:

  1. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  2. 2024Language Models with Conformal Factuality Guaranteescited by 128
  3. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 114
  4. 2024LLM-based NLG Evaluation: Current Status and Challengescited by 71
  5. 2024AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedbackcited by 25
  6. 2024Claim Verification in the Age of Large Language Models: A Surveycited by 23
  7. 2024Generative Large Language Models in Automated Fact-Checking: A Surveycited by 19
  8. 2024Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verificationcited by 13
  9. 2024ClaimVer: Explainable Claim-Level Verification and Evidence Attribution of Text Through Knowledge Graphscited by 11
  10. 2024REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropycited by 11
  11. 2025Truth over Tricks: Measuring and Mitigating Shortcut Learning in Misinformation Detectioncited by 8
  12. 2025FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMscited by 7
23 more
  1. 2026DatBench: Discriminative, Faithful, and Efficient VLM Evaluationscited by 7
  2. 2025CMIE: Combining MLLM Insights with External Evidence for Explainable Out-of-Context Misinformation Detectioncited by 6
  3. 2025Dynamic-KGQA: A Scalable Framework for Generating Adaptive Question Answering Datasetscited by 5
  4. 2025Veri-R1: Toward Precise and Faithful Claim Verification via Online Reinforcement Learningcited by 5
  5. 2025A Graph-based Verification Framework for Fact-Checkingcited by 4
  6. 2024How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Modelscited by 3
  7. 2024CrediRAG: Network-Augmented Credibility-Based Retrieval for Misinformation Detection in Redditcited by 3
  8. 2025The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factualitycited by 3
  9. 2025Speaking at the Right Level: Literacy-Controlled Counterspeech Generation with RAG-RLcited by 3
  10. 2025Exposing Citation Vulnerabilities in Generative Enginescited by 2
  11. 2025Discrepancy Detection at the Data Level: Toward Consistent Multilingual Question Answeringcited by 2
  12. 2024Recall Them All: Retrieval-Augmented Language Models for Long Object List Extraction from Long Documentscited by 1
  13. 2024LEAF: Learning and Evaluation Augmented by Fact-Checking to Improve Factualness in Large Language Modelscited by 1
  14. 2024FarExStance: Explainable Stance Detection for Farsicited by 1
  15. 2025Self-Rationalization in the Wild: A Large Scale Out-of-Distribution Evaluation on NLI-related taskscited by 1
  16. 2025CRAVE: A Conflicting Reasoning Approach for Explainable Claim Verification Using LLMscited by 1
  17. 2025Inter-Passage Verification for Multi-evidence Multi-answer QAcited by 1
  18. 2025CANDY: Benchmarking LLMs' Limitations and Assistive Potential in Chinese Misinformation Fact-Checkingcited by 1
  19. 2025The Psychology of Falsehood: A Human-Centric Survey of Misinformation Detectioncited by 1
  20. 2026Multi-LLM Adaptive Conformal Inference for Reliable LLM Responsescited by 1
  21. 2026PubMed Reasoner: Dynamic Reasoning-based Retrieval for Evidence-Grounded Biomedical Question Answeringcited by 1
  22. 2026iAgentBench: Benchmarking Sensemaking Capabilities of Information-Seeking Agents on High-Traffic Topicscited by 0
  23. 2026Is this Citation on Point?cited by 0