mlsanityData to 7 Oct 2026

Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification

arXiv 2403.04696 ↗First posted 7 Mar 2024cs.CL cs.AI cs.LGcited by 159

Not on the map. Why?

Cited by 25 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) are notorious for hallucinating, i.e., producing erroneous claims in their output. Such hallucinations can be dangerous, as occasional factual inaccuracies in the generated text might be obscured by the rest of the output being generally factually correct, making it extremely hard for the users to spot them. Current services that leverage LLMs usually do not provide any means for detecting unreliable generations. Here, we aim to bridge this gap. In particular, we propose a novel fact-checking and hallucination detection pipeline based on token-level uncertainty quantification. Uncertainty scores leverage information encapsulated in the output of a neural network or its layers to detect unreliable predictions, and we show that they can be used to fact-check the atomic claims in the LLM output. Moreover, we present a novel token-level uncertainty quantification method that removes the impact of uncertainty about what claim to generate on the current step and what surface form to use. Our method Claim Conditioned Probability (CCP) measures only the uncertainty of a particular claim value expressed by the model. Experiments on the task of biography generation demonstrate strong improvements for CCP compared to the baselines for seven LLMs and four languages. Human evaluation reveals that the fact-checking pipeline based on uncertainty quantification is competitive with a fact-checking tool that leverages external knowledge.

Key facts. Cited by 159 papers in the map (data through 7 Oct 2026). Builds on 22 of its 43 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

22 of 43 references.

References · 22

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  6. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  7. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  8. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  9. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  10. 2021A Survey on Automated Fact-Checkingcited by 376
  11. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  12. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
10 more
  1. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 210
  2. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
  3. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 193
  4. 2023Fact-Checking Complex Claims with Program-Guided Reasoningcited by 146
  5. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 130
  6. 2023Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkerscited by 86
  7. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  8. 2023Evaluating Large Language Models on Controlled Generation Taskscited by 73
  9. 2022Mutual Information Alleviates Hallucinations in Abstractive Summarizationcited by 67
  10. 2022Nonparametric Uncertainty Quantification for Single Deterministic Neural Networkcited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 63; 2026: 76.

159 papers.

Sort:

By area:

  1. 2025Deep Think with Confidencecited by 199
  2. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  3. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 107
  4. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  5. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  6. 2025A Survey of Uncertainty Estimation Methods on Large Language Modelscited by 57
  7. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  8. 2025Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?cited by 35
  9. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probescited by 35
  10. 2025Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Homecited by 33
  11. 2025Estimating LLM Uncertainty with Evidencecited by 26
  12. 2024Reference-free Hallucination Detection for Large Vision-Language Modelscited by 23
38 more
  1. 2024Atomic Calibration of LLMs in Long-Form Generationscited by 23
  2. 2024A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagationcited by 21
  3. 2024A Survey on the Honesty of Large Language Modelscited by 21
  4. 2025Entropy-Gated Branching for Efficient Test-Time Reasoningcited by 21
  5. 2025A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputscited by 20
  6. 2024LoGU: Long-form Generation with Uncertainty Expressionscited by 19
  7. 2024Rethinking Uncertainty Estimation in LLMs: A Principled Single-Sequence Measurecited by 18
  8. 2025Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Modelscited by 18
  9. 2025Hallucination Detection in LLMs with Topological Divergence on Attention Graphscited by 18
  10. 2024LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagationcited by 17
  11. 2025Correctness Assessment of Code Generated by Large Language Models Using Internal Representationscited by 17
  12. 2025LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generationscited by 17
  13. 2025Reconsidering LLM Uncertainty Estimation Methods in the Wildcited by 17
  14. 2024Loki: An Open-Source Tool for Fact Verificationcited by 16
  15. 2025CER: Confidence Enhanced Reasoning in LLMscited by 16
  16. 2025Uncertainty Quantification for Language Models: A Suite of Black-Box, White-Box, LLM Judge, and Ensemble Scorerscited by 16
  17. 2025Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Modelscited by 16
  18. 2024Core: Robust Factual Precision with Informative Sub-Claim Identificationcited by 15
  19. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 15
  20. 2025Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Headscited by 14
  21. 2024DebUnc: Improving Large Language Model Agent Communication With Uncertainty Metricscited by 13
  22. 2025HalluEntity: Benchmarking and Understanding Entity-Level Hallucination Detectioncited by 13
  23. 2025Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Modelscited by 13
  24. 2025When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQAcited by 13
  25. 2025Query-Level Uncertainty in Large Language Modelscited by 12
  26. 2025MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibrationcited by 11
  27. 2025Uncertainty-Aware Decoding with Minimum Bayes Riskcited by 10
  28. 2024Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Modelscited by 9
  29. 2024Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modelingcited by 9
  30. 2025Shadows in the Attention: Contextual Perturbation and Representation Drift in the Dynamics of Hallucination in LLMscited by 9
  31. 2025PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chainscited by 9
  32. 2025Modeling Uncertainty Trends for Timely Retrieval in Dynamic RAGcited by 8
  33. 2024Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimizationcited by 7
  34. 2025SelfReflect: Can LLMs Communicate Their Internal Answer Distribution?cited by 6
  35. 2025Span-Level Hallucination Detection for LLM-Generated Answerscited by 5
  36. 2025Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval Augmented Generationcited by 5
  37. 2025Will It Still Be True Tomorrow? Multilingual Evergreen Question Classification to Improve Trustworthy QAcited by 5
  38. 2025Uncertainty Quantification for Retrieval-Augmented Reasoningcited by 5

50 most cited of 159 All 159 (JSON)