mlsanityData to 14 Oct 2026

Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Models

arXiv 2502.11028 ↗First posted 16 Feb 2025cs.CL cs.AIcited by 40

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) show remarkable proficiency in natural language tasks, yet their frequent overconfidence-misalignment between predicted confidence and true correctness-poses significant risks in critical decision-making applications. We present a comprehensive analysis on calibration in LLMs across nine LLMs and three factual Question-Answering (QA) datasets, systematically comparing standard free-generation settings against structured distractor-augmented prompts. Our evaluation reveals that explicitly incorporating distractors can substantially mitigate miscalibration, achieving relative accuracy improvements up to 460% and ECE reductions up to 90%. Despite general trends, we uncover nuanced findings: large RLHF-tuned models display inherent calibration strengths but can paradoxically suffer increased miscalibration on easier queries, whereas smaller models benefit disproportionately from distractor prompts but remain significantly miscalibrated. Through detailed analyses across question types, we identify persistent calibration failures, particularly in person-based queries. We conclude with concrete recommendations-targeted fine-tuning, structured prompting, and strategic model choice-to ensure reliable, trustworthy LLM deployments.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Builds on 31 of its 44 references that are in the map. First posted on arXiv 16 Feb 2025.

Builds on

31 of 44 references.

References · 31

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  8. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  10. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  11. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  12. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
19 more
  1. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  2. 2016Language Modeling with Gated Convolutional Networkscited by 702
  3. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  4. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  5. 2024Measuring short-form factuality in large language modelscited by 373
  6. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
  7. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 210
  8. 2024Taming Overconfidence in LLMs: Reward Calibration in RLHFcited by 112
  9. 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
  10. 2023On the Calibration of Large Language Models and Alignmentcited by 73
  11. 2024Calibrating Large Language Models Using Their Generations Onlycited by 65
  12. 2022A Close Look into the Calibration of Pre-trained Language Modelscited by 57
  13. 2023LitCab: Lightweight Language Model Calibration over Short- and Long-form Responsescited by 44
  14. 2021FaVIQ: FAct Verification from Information-seeking Questionscited by 36
  15. 2022On the Calibration of Pre-trained Language Models using Mixup Guided by Area Under the Margin and Saliencycited by 31
  16. 2023A Diachronic Perspective on User Trust in AI under Uncertaintycited by 23
  17. 2024On Calibration of LLM-based Guard Models for Reliable Content Moderationcited by 23
  18. 2023A Study on the Calibration of In-context Learningcited by 22
  19. 2023FIRE: Food Image to REcipe generationcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 17; 2026: 23.

40 papers.

Sort:

By area:

  1. 2025LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on Challenging Queriescited by 17
  2. 2025Calibrating Verbalized Confidence with Self-Generated Distractorscited by 14
  3. 2025ParallelMuse: Agentic Parallel Thinking for Deep Information Seekingcited by 14
  4. 2026Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewardscited by 14
  5. 2025Cascaded Language Models for Cost-effective Human-AI Decision-Makingcited by 11
  6. 2025When Two LLMs Debate, Both Think They'll Wincited by 10
  7. 2025Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Modelscited by 10
  8. 2026Hidden States as Early Signals: Step-level Trace Evaluation and Pruning for Efficient Test-Time Scalingcited by 10
  9. 2025C2GSPG: Confidence-calibrated Group Sequence Policy Gradient towards Self-aware Reasoningcited by 8
  10. 2025Robust Sampling for Active Statistical Inferencecited by 6
  11. 2026Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigationcited by 6
  12. 2025Automated Formalization via Conceptual Retrieval-Augmented LLMscited by 5
28 more
  1. 2025BrowseConf: Confidence-Guided Test-Time Scaling for Web Agentscited by 5
  2. 2025Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitationcited by 5
  3. 2026I-CALM: Incentivizing Confidence-Aware Abstention for LLM Selective Answeringcited by 5
  4. 2025When an LLM is apprehensive about its answers -- and when its uncertainty is justifiedcited by 4
  5. 2025Another Turn, Better Output? A Turn-Wise Analysis of Iterative LLM Promptingcited by 3
  6. 2025Model-agnostic Selective Labeling with Provable Statistical Guaranteescited by 3
  7. 2026Know More, Know Clearer: A Meta-Cognitive Framework for Knowledge Augmentation in Large Language Modelscited by 3
  8. 2026Evaluating Confidence-Gated Retrieval with Matched Trajectory Replaycited by 3
  9. 2025Conf-Profile: A Confidence-Driven Reasoning Paradigm for Label-Free User Profilingcited by 2
  10. 2025Bounds of Chain-of-Thought Robustness: Reasoning Steps, Embed Norms, and Beyondcited by 2
  11. 2026Hidden State Poisoning Attacks against Mamba-based Language Modelscited by 2
  12. 2026Process Supervision of Confidence Margin for Calibrated LLM Reasoningcited by 2
  13. 2026Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answercited by 2
  14. 2025Contemplative Artificial Intelligencecited by 1
  15. 2026Completing Missing Annotation: Multi-Agent Debate for Accurate and Scalable Relevant Assessment for IR Benchmarkscited by 1
  16. 2026DMAP: A Distribution Map for Textcited by 1
  17. 2026SHOE: Semantic HOI Open-Vocabulary Evaluation Metriccited by 1
  18. 2026Capability Self-Assessment in Large Language Modelscited by 1
  19. 2026FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertaintycited by 1
  20. 2026CALIBER: Calibrating Confidence Before and After Reasoning in Language Modelscited by 1
  21. 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
  22. 2026The Personalization Mirage: How LLMs Fabricate User Profiles, and Why Self-Monitoring Misleadscited by 1
  23. 2026The Dunning-Kruger Effect in Large Language Models: An Empirical Study of Confidence Calibrationcited by 0
  24. 2026Task-Aware Calibration: Provably Optimal Decoding in LLMscited by 0
  25. 2026Inducing Artificial Uncertainty in Language Modelscited by 0
  26. 2026LLM-as-a-Discriminator: When Synthetic Tables Still Look Realcited by 0
  27. 2026From token probabilities to calibrated confidence: An empirical study of mathematical question answeringcited by 0
  28. 2026When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Informationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ConfidenceBench: Evaluating Confidence Calibration in Large Language Models
  2. 2024Calibrating the Confidence of Large Language Models by Eliciting Fidelity
  3. 2025Trained on Tokens, Calibrated on Concepts: The Emergence of Semantic Calibration in LLMs
  4. 2026A Semantic-Sampling Framework for Evaluating Calibration in Open-Ended Question Answering
  5. 2025SteerConf: Steering LLMs for Confidence Elicitation
  6. 2026Closing the Confidence-Faithfulness Gap in Large Language Models
  7. 2025Human-Alignment and Calibration of Inference-Time Uncertainty in Large Language Models
  8. 2024What Large Language Models Know and What People Think They Know