mlsanityData to 14 Oct 2026

A Survey of Uncertainty Estimation Methods on Large Language Models

arXiv 2503.00172 ↗First posted 28 Feb 2025cs.CLcited by 62

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have demonstrated remarkable capabilities across various tasks. However, these models could offer biased, hallucinated, or non-factual responses camouflaged by their fluency and realistic appearance. Uncertainty estimation is the key method to address this challenge. While research efforts in uncertainty estimation are ramping up, there is a lack of comprehensive and dedicated surveys on LLM uncertainty estimation. This survey presents four major avenues of LLM uncertainty estimation. Furthermore, we perform extensive experimental evaluations across multiple methods and datasets. At last, we provide critical and promising future directions for LLM uncertainty estimation.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Builds on 42 of its 57 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

42 of 57 references.

References · 42

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2023A Survey of Large Language Modelscited by 2,576
  8. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  9. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  10. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  11. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  12. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
30 more
  1. 2023Large Language Modelscited by 780
  2. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  3. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  4. 2022Large Language Models Can Self-Improvecited by 664
  5. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 479
  6. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  7. 2024Measuring short-form factuality in large language modelscited by 373
  8. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
  9. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
  10. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 211
  11. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 197
  12. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 162
  13. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 159
  14. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 139
  15. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 131
  16. 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 122
  17. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 114
  18. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 99
  19. 2023Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Modelscited by 91
  20. 2024LUQ: Long-text Uncertainty Quantification for LLMscited by 81
  21. 2024Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approachcited by 76
  22. 2024Calibrating Large Language Models with Sample Consistencycited by 73
  23. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  24. 2019This Email Could Save Your Life: Introducing the Task of Email Subject Line Generationcited by 58
  25. 2024A Survey of Uncertainty Estimation in LLMs: Theory Meets Practicecited by 55
  26. 2024Distinguishing the Knowable from the Unknowable with Language Modelscited by 47
  27. 2024MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMscited by 46
  28. 2024Multicalibration for Confidence Scoring in LLMscited by 46
  29. 2024Uncertainty Quantification for In-Context Learning of Large Language Modelscited by 45
  30. 2024Uncertainty in Language Models: Assessment through Rank-Calibrationcited by 41

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 50.

62 papers.

Sort:

By area:

  1. 2025MUR: Momentum Uncertainty guided Reasoning for Large Language Modelscited by 17
  2. 2025MetaFaith: Faithful Natural Language Uncertainty Expression in LLMscited by 16
  3. 2025Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMscited by 13
  4. 2025PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chainscited by 10
  5. 2026Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantificationcited by 10
  6. 2026Every Response Counts: Quantifying Uncertainty of LLM-based Multi-Agent Systems through Tensor Decompositioncited by 7
  7. 2026NOVA: NOise-aware Verbal Confidence CAlibration for Robust Large Language Models in RAG Systemscited by 6
  8. 2026It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertaintycited by 6
  9. 2026Verbalizing LLM's Higher-order Uncertainty via Imprecise Probabilitiescited by 5
  10. 2026I-CALM: Incentivizing Confidence-Aware Abstention for LLM Selective Answeringcited by 5
  11. 2025Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Reviewcited by 4
  12. 2025CritiCal: Can Critique Help LLM Uncertainty or Confidence Calibration?cited by 4
38 more
  1. 2026AI Alignment Breaks at the Edgecited by 4
  2. 2026Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Modelscited by 4
  3. 2026Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMscited by 4
  4. 2026STRUCTSURVEY: Structured Agentic Retrieval for Automated Survey Paper Generationcited by 4
  5. 2025Don't Throw Away Your Beams: Improving Consistency-based Uncertainties in LLMs via Beam Searchcited by 3
  6. 2026Uncertainty Propagation in LLM-Based Systemscited by 3
  7. 2026Uncertainty Decomposition for Clarification Seeking in LLM Agentscited by 3
  8. 2025Toward Efficient Uncertainty in LLMs through Evidential Knowledge Distillationcited by 2
  9. 2025FinZero: Launching Multi-modal Financial Time Series Forecast with Large Reasoning Modelcited by 2
  10. 2026LLM Reasoning Predicts When Models Are Right: Evidence from Coding Classroom Discoursecited by 2
  11. 2026Furina: Fragmented Uncertainty-Driven Refusal Instability Attackcited by 2
  12. 2026Can LLMs Use Linguistic Uncertainty Markers to Reliably Reflect Intrinsic Confidence?cited by 2
  13. 2026A Systematic Evaluation of Black-Box Uncertainty Estimation Methods for Large Language Modelscited by 2
  14. 2026Aligning Language Models with Selective Predictioncited by 2
  15. 2026Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agentscited by 2
  16. 2025MUCH: A Multilingual Claim Hallucination Benchmarkcited by 1
  17. 2025Token-Level Marginalization for Multi-Label LLM Classifierscited by 1
  18. 2026Efficient Epistemic Uncertainty Estimation for Large Language Models via Knowledge Distillationcited by 1
  19. 2026Managing Uncertainty in LLM-based Multi-Agent System Operationcited by 1
  20. 2026Think Twice Before You Write -- an Entropy-based Decoding Strategy to Enhance LLM Reasoningcited by 1
  21. 2026Position: Logical Soundness is not a Reliable Criterion for Neurosymbolic Fact-Checking with LLMscited by 1
  22. 2026InfiniteScienceGym: An Unbounded, Procedurally-Generated Benchmark for Scientific Analysiscited by 1
  23. 2026Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Modelscited by 1
  24. 2026Belief-Guided Inference Control for Large Language Model Services via Verifiable Observationscited by 1
  25. 2026Geometric Deviation as an Unsupervised Pre-Generation Reliability Signal: Probing LLM Representations for Answerabilitycited by 1
  26. 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
  27. 2025Bridging Subjective and Objective QoE: Operator-Level Aggregation Using LLM-Based Comment Analysis and Network MOS Comparisoncited by 0
  28. 2026Towards a Science of Collective AI: LLM-based Multi-Agent Systems Need a Transition from Blind Trial-and-Error to Rigorous Sciencecited by 0
  29. 2026Vision Verification Enhanced Fusion of VLMs for Efficient Visual Reasoningcited by 0
  30. 2026A Formal Framework for Uncertainty Analysis of Text Generation with Large Language Modelscited by 0
  31. 2026Identifying Influential N-grams in Confidence Calibration via Regression Analysiscited by 0
  32. 2026Why Don't You Know? Evaluating the Impact of Uncertainty Sources on Uncertainty Quantification in LLMscited by 0
  33. 2026Confidence Estimation in Automatic Short Answer Grading with LLMscited by 0
  34. 2026Epistemic Uncertainty for Test-Time Discoverycited by 0
  35. 2026ECUASn: A family of metrics for principled evaluation of uncertainty-augmented systemscited by 0
  36. 2026Entropy Distribution as a Fingerprint for Hallucinations in Generative Modelscited by 0
  37. 2026Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Modelscited by 0
  38. 2026Beyond Semantic Equivalence: Logical Graphs for LLM Uncertainty Quantificationcited by 0

50 most cited of 62 All 62 (JSON)