mlsanityData to 7 Oct 2026

From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered

arXiv 2506.07461 ↗First posted 9 Jun 2025cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are increasingly assisting users in the real world, yet their reliability remains a concern. Uncertainty quantification (UQ) has been heralded as a tool to enhance human-LLM collaboration by enabling users to know when to trust LLM predictions. We argue that current practices for uncertainty quantification in LLMs are not optimal for developing useful UQ for human users making decisions in real-world tasks. Through an analysis of 40 LLM UQ methods, we identify three prevalent practices hindering the community's progress toward its goal of benefiting downstream users: 1) evaluating on benchmarks with low ecological validity; 2) considering only epistemic uncertainty; and 3) optimizing metrics that are not necessarily indicative of downstream utility. For each issue, we propose concrete user-centric practices and research directions that LLM UQ researchers should consider. Instead of hill-climbing on unrepresentative tasks using imperfect metrics, we argue that the community should adopt a more human-centered approach to LLM uncertainty quantification.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 101 of its 130 references that are in the map. First posted on arXiv 9 Jun 2025.

Builds on

101 of 130 references.

References · 101

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  5. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  6. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  7. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  8. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  9. 2017Selective Classification for Deep Neural Networkscited by 799
  10. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  11. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  12. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
38 more
  1. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  2. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  3. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  4. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  5. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  6. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  7. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  8. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  9. 2022Challenges and Strategies in Cross-Cultural NLPcited by 216
  10. 2024Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoderscited by 213
  11. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 211
  12. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 209
  13. 2020Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthycited by 182
  14. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  15. 2020What Can We Learn from Collective Human Opinions on Natural Language Inference Data?cited by 148
  16. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 147
  17. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  18. 2020Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systemscited by 138
  19. 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 121
  20. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  21. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  22. 2023Bayesian Low-rank Adaptation for Large Language Modelscited by 112
  23. 2025Confidence Improves Self-Consistency in LLMscited by 111
  24. 2021WebQA: Multihop and Multimodal QAcited by 109
  25. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
  26. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 98
  27. 2023CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotationcited by 93
  28. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  29. 2024Large language model validity via enhanced conformal prediction methodscited by 91
  30. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
  31. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 84
  32. 2023A Survey on Uncertainty Quantification Methods for Deep Learningcited by 78
  33. 2020A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scorescited by 74
  34. 2024Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approachcited by 73
  35. 2022CLAM: Selective Clarification for Ambiguous Questions with Generative Language Modelscited by 66
  36. 2023Selectively Answering Ambiguous Questionscited by 63
  37. 2024Calibrating Large Language Models Using Their Generations Onlycited by 63
  38. 2023Smooth ECE: Principled Reliability Diagrams via Kernel Smoothingcited by 59

50 most cited of 101 All 101 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 14.

21 papers.

Sort:

By area:

  1. 2025Trace Length is a Simple Uncertainty Signal in Reasoning Modelscited by 16
  2. 2025The Illusion of Certainty: Uncertainty Quantification for LLMs Fails under Ambiguitycited by 15
  3. 2026Are LLM Decisions Faithful to Verbal Confidence?cited by 6
  4. 2025Can Large Language Models Express Uncertainty Like Human?cited by 5
  5. 2025On the Robustness of Verbal Confidence of LLMs in Adversarial Attackscited by 3
  6. 2025Localized Calibrated Uncertainty in Code Language Modelscited by 3
  7. 2026SAFECAST: Robust Failure Detection for VLA Policies with Contrast-Set Training and Calibrationcited by 2
  8. 2026From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Modelscited by 1
  9. 2026Uncertainty and Fairness Awareness in LLM-Based Recommendation Systemscited by 1
  10. 2026Learning to Decide with AI Assistance under Human-Alignmentcited by 1
  11. 2026Testing LLM Arithmetic Reasoning Generalization with Automatic Numeric-Remapping Attackscited by 1
  12. 2025PaTAS: A Framework for Trust Propagation in Neural Networks Using Subjective Logiccited by 0
9 more
  1. 2025Knowing Your Uncertainty -- On the application of LLM in social sciencescited by 0
  2. 2026From Performance to Purpose: A Sociotechnical Taxonomy for Evaluating Large Language Model Utilitycited by 0
  3. 2026Position: Uncertainty Quantification in LLMs is Just Unsupervised Clusteringcited by 0
  4. 2026Evaluating the Relevance of Uncertainty Estimators for LLM Hallucinationcited by 0
  5. 2026Localizing Input Uncertainty Quantification for Large Language Models via Shapley Valuescited by 0
  6. 2026Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Modelscited by 0
  7. 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applicationscited by 0
  8. 2026Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMscited by 0
  9. 2026Characterizing Overconfident Failure in LLM-Based Code Generationcited by 0