mlsanityData to 14 Oct 2026

Calibrating Large Language Models with Sample Consistency

arXiv 2402.13904 ↗First posted 21 Feb 2024cs.CLcited by 73

Not on the map. Why?

Abstract (arXiv)

Accurately gauging the confidence level of Large Language Models' (LLMs) predictions is pivotal for their reliable application. However, LLMs are often uncalibrated inherently and elude conventional calibration techniques due to their proprietary nature and massive scale. In this work, we explore the potential of deriving confidence from the distribution of multiple randomly sampled model generations, via three measures of consistency. We perform an extensive evaluation across various open and closed-source models on nine reasoning datasets. Results show that consistency-based calibration methods outperform existing post-hoc approaches. Meanwhile, we find that factors such as intermediate explanations, model scaling, and larger sample sizes enhance calibration, while instruction-tuning makes calibration more difficult. Moreover, confidence scores obtained from consistency have the potential to enhance model performance. Finally, we offer practical guidance on choosing suitable consistency metrics for calibration, tailored to the characteristics of various LMs.

Key facts. Cited by 73 papers in the map (data through 14 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 21 Feb 2024.

Builds on

30 of 41 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2017On Calibration of Modern Neural Networkscited by 4,960
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  10. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  11. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  12. 2022Language Models (Mostly) Know What They Knowcited by 1,856
18 more
  1. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  2. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  3. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  4. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  5. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  6. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  7. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  8. 2016Solving General Arithmetic Word Problemscited by 360
  9. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  10. 2023Faithful Chain-of-Thought Reasoningcited by 288
  11. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
  12. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 210
  13. 2019CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Textcited by 162
  14. 2021Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoningcited by 85
  15. 2022A Close Look into the Calibration of Pre-trained Language Modelscited by 57
  16. 2022Automatic Generation of Socratic Subquestions for Teaching Math Word Problemscited by 43
  17. 2022Detection of Word Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimationcited by 32
  18. 2023SCREWS: A Modular Framework for Reasoning with Revisionscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 16; 2025: 26; 2026: 31.

73 papers.

Sort:

By area:

  1. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 232
  2. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  3. 2025Confidence Improves Self-Consistency in LLMscited by 113
  4. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 105
  5. 2025The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systemscited by 72
  6. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 66
  7. 2025A Survey of Uncertainty Estimation Methods on Large Language Modelscited by 62
  8. 2025Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoningcited by 31
  9. 2024DyKnow: Dynamically Verifying Time-Sensitive Factual Knowledge in LLMscited by 24
  10. 2024Do Not Design, Learn: A Trainable Scoring Function for Uncertainty Estimation in Generative LLMscited by 23
  11. 2024A Survey on the Honesty of Large Language Modelscited by 23
  12. 2024HonestLLM: Toward an Honest and Helpful Large Language Modelcited by 18
38 more
  1. 2025Semantic Energy: Detecting LLM Hallucination Beyond Entropycited by 18
  2. 2024Mirror-Consistency: Harnessing Inconsistency in Majority Votingcited by 17
  3. 2025Alignment for Efficient Tool Calling of Large Language Modelscited by 17
  4. 2025LLMs Can Teach Themselves to Better Predict the Futurecited by 14
  5. 2026SCOPE: Selective Conformal Optimized Pairwise LLM Judgingcited by 14
  6. 2024QA-Calibration of Language Model Confidence Scorescited by 13
  7. 2024UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Modelscited by 13
  8. 2025Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregationcited by 11
  9. 2025MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibrationcited by 11
  10. 2025PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chainscited by 10
  11. 2025Calibration Across Layers: Understanding Calibration Evolution in LLMscited by 9
  12. 2025Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimationcited by 9
  13. 2026Optimal Bayesian Stopping for Efficient Inference of Consistent LLM Answerscited by 8
  14. 2024Navigating Noisy Feedback: Enhancing Reinforcement Learning with Error-Prone Language Modelscited by 7
  15. 2025Conformal Information Pursuit for Interactively Guiding Large Language Modelscited by 7
  16. 2024SIKeD: Self-guided Iterative Knowledge Distillation for mathematical reasoningcited by 6
  17. 2025B-score: Detecting biases in large language models using response historycited by 6
  18. 2025Prompt4Trust: A Reinforcement Learning Prompt Augmentation Framework for Clinically-Aligned Confidence Calibration in Multimodal Large Language Modelscited by 6
  19. 2025Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impactscited by 6
  20. 2026NOVA: NOise-aware Verbal Confidence CAlibration for Robust Large Language Models in RAG Systemscited by 6
  21. 2026Same Answer, Different Confidence: Protocol Sensitivity in LLM Confidence Calibrationcited by 6
  22. 2025Do Retrieval Augmented Language Models Know When They Don't Know?cited by 5
  23. 2026The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Modelscited by 5
  24. 2025Delusions of Large Language Modelscited by 4
  25. 2025Generalization of Fine-Tuned Uncertainty Communication and Metacognition in Large Language Modelscited by 4
  26. 2025Taming Object Hallucinations with Verified Atomic Confidence Estimationcited by 4
  27. 2025Are Economists Always More Introverted? Analyzing Consistency in Persona-Assigned LLMscited by 3
  28. 2025All Roads Lead to Rome: Graph-Based Confidence Estimation for Large Language Model Reasoningcited by 3
  29. 2025Systematic Evaluation of Uncertainty Estimation Methods in Large Language Modelscited by 3
  30. 2026Evaluating and Calibrating LLM Confidence on Questions with Multiple Correct Answerscited by 3
  31. 2026Calibrating Model-Based Evaluation Metrics for Summarizationcited by 3
  32. 2026Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generationcited by 3
  33. 2024Gradual Learning: Optimizing Fine-Tuning with Partially Mastered Knowledge in Large Language Modelscited by 2
  34. 2025Reshaping MOFs text mining with a dynamic multi-agents framework of large language modelcited by 2
  35. 2025SCOPE: Stochastic and Counterbiased Option Placement for Evaluating Large Language Modelscited by 2
  36. 2026LLM-Confidence Reranker: A Training-Free Approach for Enhancing Retrieval-Augmented Generation Systemscited by 2
  37. 2026Parallel Test-Time Scaling with Multi-Sequence Verifierscited by 2
  38. 2026Recursive Language Models Meet Uncertainty: The Surprising Effectiveness of Self-Reflective Program Search for Long Contextcited by 2

50 most cited of 73 All 73 (JSON)