mlsanityData to 14 Oct 2026

Revisiting Uncertainty Estimation and Calibration of Large Language Models

arXiv 2505.23854 ↗First posted 29 May 2025cs.CL cs.AI cs.LGcited by 23

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

As large language models (LLMs) are increasingly deployed in high-stakes applications, robust uncertainty estimation is essential for ensuring the safe and trustworthy deployment of LLMs. We present the most comprehensive study to date of uncertainty estimation in LLMs, evaluating 80 models spanning open- and closed-source families, dense and Mixture-of-Experts (MoE) architectures, reasoning and non-reasoning modes, quantization variants and parameter scales from 0.6B to 671B. Focusing on three representative black-box single-pass methods, including token probability-based uncertainty (TPU), numerical verbal uncertainty (NVU), and linguistic verbal uncertainty (LVU), we systematically evaluate uncertainty calibration and selective classification using the challenging MMLU-Pro benchmark, which covers both reasoning-intensive and knowledge-based tasks. Our results show that LVU consistently outperforms TPU and NVU, offering stronger calibration and discrimination while being more interpretable. We also find that high accuracy does not imply reliable uncertainty, and that model scale, post-training, reasoning ability and quantization all influence estimation performance. Notably, LLMs exhibit better uncertainty estimates on reasoning tasks than on knowledge-heavy ones, and good calibration does not necessarily translate to effective error ranking. These findings highlight the need for multi-perspective evaluation and position LVU as a practical tool for improving the reliability of LLMs in real-world settings.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 33 of its 54 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

33 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Qwen2.5 Technical Report19 Dec

Other references · 32

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  9. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  10. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  11. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  12. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
20 more
  1. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  2. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  3. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  4. 2019Measuring Calibration in Deep Learningcited by 446
  5. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  6. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  7. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
  8. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 246
  9. 2023Large Language Models in Law: A Surveycited by 128
  10. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 95
  11. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 87
  12. 2024MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notescited by 80
  13. 2024Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Taskscited by 78
  14. 2024Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approachcited by 76
  15. 2023On the Calibration of Large Language Models and Alignmentcited by 73
  16. 2024A Survey of Uncertainty Estimation in LLMs: Theory Meets Practicecited by 55
  17. 2023Identifying and Mitigating Vulnerabilities in LLM-Integrated Applicationscited by 54
  18. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  19. 2025Mitigating Object Hallucinations in Large Vision-Language Models via Attention Calibrationcited by 23
  20. 2024Perceptions of Linguistic Uncertainty by Language Models and Humanscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 17.

23 papers.

Sort:

By area:

  1. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 20
  2. 2025Trace Length is a Simple Uncertainty Signal in Reasoning Modelscited by 19
  3. 2025Semantic Energy: Detecting LLM Hallucination Beyond Entropycited by 18
  4. 2025Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missingcited by 14
  5. 2026Causal Evidence that Language Models use Confidence to Drive Behaviorcited by 10
  6. 2026Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agentscited by 8
  7. 2026Hallucinations Undermine Trust; Metacognition is a Way Forwardcited by 5
  8. 2025Reasoning's Razor: Reasoning Improves Accuracy but Can Hurt Recall at Critical Operating Points in Safety and Hallucination Detectioncited by 3
  9. 2025Toward Efficient Uncertainty in LLMs through Evidential Knowledge Distillationcited by 2
  10. 2026Believe Your Model: Distribution-Guided Confidence Calibrationcited by 2
  11. 2026Retrieval-Augmented Linguistic Calibrationcited by 2
  12. 2026Empirical Characterization of Inference-Time Elicited Probability Transformations in Large Language Modelscited by 1
11 more
  1. 2026Fork-Think with Confidencecited by 1
  2. 2025The Architecture of Trust: A Framework for AI-Augmented Real Estate Valuation in the Era of Structured Datacited by 0
  3. 2026Do Small Language Models Know When They're Wrong? Confidence-Based Cascade Scoring for Educational Assessmentcited by 0
  4. 2026"I Don't Know" -- Towards Appropriate Trust with Certainty-Aware Retrieval Augmented Generationcited by 0
  5. 2026Epistemic Uncertainty for Test-Time Discoverycited by 0
  6. 2026Gradient-Guided Reward Optimization for Inference-time Alignmentcited by 0
  7. 2026When Uncertainty Isn't Enough: An Empirical Study of Self-Correction in Code Generationcited by 0
  8. 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Studycited by 0
  9. 2026On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Modelscited by 0
  10. 2026Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Modelscited by 0
  11. 2026Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Maskingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Simple Yet Effective: An Information-Theoretic Approach to Multi-LLM Uncertainty Quantification
  2. 2026Dissecting Training-Free Uncertainty Estimation in Multimodal Large Language Models
  3. 2026The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models
  4. 2024Rethinking the Uncertainty: A Critical Review and Analysis in the Era of Large Language Models
  5. 2026Towards Reliable Truth-Aligned Uncertainty Estimation in Large Language Models
  6. 2024UBench: Benchmarking Uncertainty in Large Language Models with Multiple Choice Questions
  7. 2026Rethinking Uncertainty Evaluation in Large Language Models
  8. 2026From Entropy to Calibrated Uncertainty: Training Language Models to Reason About Uncertainty