mlsanityData to 14 Oct 2026

Overconfidence is Key: Verbalized Uncertainty Evaluation in Large Language and Vision-Language Models

arXiv 2405.02917 ↗First posted 5 May 2024cs.CV cs.CL cs.LGcited by 54

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Language and Vision-Language Models (LLMs/VLMs) have revolutionized the field of AI by their ability to generate human-like text and understand images, but ensuring their reliability is crucial. This paper aims to evaluate the ability of LLMs (GPT4, GPT-3.5, LLaMA2, and PaLM 2) and VLMs (GPT4V and Gemini Pro Vision) to estimate their verbalized uncertainty via prompting. We propose the new Japanese Uncertain Scenes (JUS) dataset, aimed at testing VLM capabilities via difficult queries and object counting, and the Net Calibration Error (NCE) to measure direction of miscalibration. Results show that both LLMs and VLMs have a high calibration error and are overconfident most of the time, indicating a poor capability for uncertainty estimation. Additionally we develop prompts for regression tasks, and we show that VLMs have poor calibration when producing mean/standard deviation and 95% confidence intervals.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 7 of its 18 references that are in the map. First posted on arXiv 5 May 2024.

Builds on

7 of 18 references.

References · 7

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  6. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  7. 2021I Find Your Lack of Uncertainty in Computer Vision Disturbingcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 18; 2026: 30.

54 papers.

Sort:

By area:

  1. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  2. 2025From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centeredcited by 21
  3. 2024Prune 'n Predict: Optimizing LLM Decision-making with Conformal Predictioncited by 18
  4. 2026Agentic Confidence Calibrationcited by 15
  5. 2025Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missingcited by 14
  6. 2025Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Taskscited by 10
  7. 2025Do Large Language Models Know What They Are Capable Of?cited by 10
  8. 2026The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillationcited by 9
  9. 2026Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoringcited by 9
  10. 2025Calibrating Uncertainty Quantification of Multi-Modal LLMs using Groundingcited by 8
  11. 2026Closing the Confidence-Faithfulness Gap in Large Language Modelscited by 8
  12. 2025CaRT: Teaching LLM Agents to Know When They Know Enoughcited by 7
38 more
  1. 2026Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigationcited by 6
  2. 2024Enhancing Trust in Large Language Models via Uncertainty-Calibrated Fine-Tuningcited by 5
  3. 2025Uncertainty in Action: Confidence Elicitation in Embodied Agentscited by 5
  4. 2025Visual Instruction Bottleneck Tuningcited by 5
  5. 2025Outlier Synthesis via Hamiltonian Monte Carlo for Out-of-Distribution Detectioncited by 4
  6. 2025Refine and Align: Confidence Calibration through Multi-Agent Interaction in VQAcited by 4
  7. 2026Instinct vs. Reflection: Unifying Token and Verbalized Confidence in Multimodal Large Modelscited by 4
  8. 2025Are vision language models robust to uncertain inputs?cited by 3
  9. 2025On the Robustness of Verbal Confidence of LLMs in Adversarial Attackscited by 3
  10. 2025Do We Really Even Need Data? A Modern Look at Drawing Inference with Predicted Datacited by 3
  11. 2024The Dilemma of Uncertainty Estimation for General Purpose AI in the EU AI Actcited by 2
  12. 2024Reflexive Guidance: Improving OoDD in Vision-Language Models via Self-Guided Image-Adaptive Concept Generationcited by 2
  13. 2025Prune-Then-Plan: Step-Level Calibration for Stable Frontier Exploration in Embodied Question Answeringcited by 2
  14. 2024Pretraining with random noise for uncertainty calibrationcited by 1
  15. 2025Uncertainty Quantification for Motor Imagery BCI -- Machine Learning vs. Deep Learningcited by 1
  16. 2026Similarity-as-Evidence: Calibrating Overconfident VLMs for Interpretable and Label-Efficient Medical Active Learningcited by 1
  17. 2026SIEVES: Selective Prediction Generalizes through Visual Evidence Scoringcited by 1
  18. 2026Calibrating LLMs with Semantic-level Rewardcited by 1
  19. 2026Capability Self-Assessment in Large Language Modelscited by 1
  20. 2025Gauging Overprecision in LLMs: An Empirical Studycited by 0
  21. 2025Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibrationcited by 0
  22. 2025Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanationscited by 0
  23. 2026ViTaB-A: Evaluating Multimodal Large Language Models on Visual Table Attributioncited by 0
  24. 2026The Dunning-Kruger Effect in Large Language Models: An Empirical Study of Confidence Calibrationcited by 0
  25. 2026Edge Reliability Gap in Vision-Language Models: Quantifying Failure Modes of Compressed VLMs Under Visual Corruptioncited by 0
  26. 2026Less Data Approximates More: Earning Faithful Confidence in High-Stakes Domainscited by 0
  27. 2026Reliable AI Needs to Externalize Implicit Knowledge: A Human-AI Collaboration Perspectivecited by 0
  28. 2026PinPoint: Prompting with Informative Interior Pointscited by 0
  29. 2026VLM-GLoc: Vision-Language Model Enhanced Monte Carlo Localization for Robust Semantic Global Localization in Cluttered Quasi-Static Environmentscited by 0
  30. 2026Visual Semantic Entropy: Do Vision Language Models Recognize Visual Ambiguity?cited by 0
  31. 2026Code-MUE: Measuring Code LLMs' Uncertainty through Execution-based Semantic Interaction Graphscited by 0
  32. 2026Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradationcited by 0
  33. 2026Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradationcited by 0
  34. 2026When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Informationcited by 0
  35. 2026Object Counting Across Modalities: Taxonomies, Benchmarks, Applications, and Open Challengescited by 0
  36. 2026MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agentscited by 0
  37. 2026Can VLMs Reliably Assess Sidewalk Accessibility Attributes from Pedestrian-Level Imagery?cited by 0
  38. 2026Pinocchio: Fast Uncertainty Estimates for Black-Box Language Modelscited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Models
  2. 2024Uncertainty-Aware Evaluation for Vision-Language Models
  3. 2025Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models
  4. 2025The Art of Saying "Maybe": A Conformal Lens for Uncertainty Benchmarking in VLMs
  5. 2024Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models
  6. 2025Object-Level Verbalized Confidence Calibration in Vision-Language Models via Semantic Perturbation
  7. 2026FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty
  8. 2026Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation