mlsanityData to 14 Oct 2026

Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models

arXiv 2412.14660 ↗First posted 19 Dec 2024cs.CV cs.AI cs.CLcited by 19

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal large language models (MLLMs) combine visual and textual data for tasks such as image captioning and visual question answering. Proper uncertainty calibration is crucial, yet challenging, for reliable use in areas like healthcare and autonomous driving. This paper investigates representative MLLMs, focusing on their calibration across various scenarios, including before and after visual fine-tuning, as well as before and after multimodal training of the base LLMs. We observed miscalibration in their performance, and at the same time, no significant differences in calibration across these scenarios. We also highlight how uncertainty differs between text and images and how their integration affects overall uncertainty. To better understand MLLMs' miscalibration and their ability to self-assess uncertainty, we construct the IDK (I don't know) dataset, which is key to evaluating how they handle unknowns. Our findings reveal that MLLMs tend to give answers rather than admit uncertainty, but this self-assessment improves with proper prompt adjustments. Finally, to calibrate MLLMs and enhance model reliability, we propose techniques such as temperature scaling and iterative prompt optimization. Our results provide insights into improving MLLMs for effective and responsible deployment in multimodal applications. Code and IDK dataset: https://github.com/hfutml/Calibration-MLLM.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 46 references that are in the map. First posted on arXiv 19 Dec 2024.

Builds on

14 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Aligning Large Multimodal Models with Factually Augmented RLHF25 Sep
  2. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul

Other references · 12

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  4. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  5. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  6. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  7. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  8. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  9. 2024Can AI Assistants Know What They Don't Know?cited by 50
  10. 2023Investigating Uncertainty Calibration of Aligned Language Models under the Multiple-Choice Settingcited by 20
  11. 2023Exploring Transferability of Multimodal Adversarial Samples for Vision-Language Pre-training Models with Contrastive Learningcited by 16
  12. 2023Deep sub-ensembles meets quantile regression: uncertainty-aware imputation for time seriescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 13.

19 papers.

Sort:

By area:

  1. 2025Evaluating Uncertainty and Quality of Vision-Language-Action-enabled Robotscited by 17
  2. 2025Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Modelscited by 16
  3. 2026Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volumecited by 8
  4. 2025Uncertainty-o: One Model-agnostic Framework for Unveiling Uncertainty in Large Multimodal Modelscited by 5
  5. 2025Explicit Abstention Knobs for Predictable Reliability in Video Question Answeringcited by 3
  6. 2026Don't Blink: Evidence Collapse during Multimodal Reasoningcited by 2
  7. 2025Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoningcited by 1
  8. 2025Analysis of Image-and-Text Uncertainty Propagation in Multimodal Large Language Models with Cardiac MR-Based Applicationscited by 1
  9. 2026FaithSCAN: Model-Driven Single-Pass Hallucination Detection for Faithful Visual Question Answeringcited by 1
  10. 2026When Does Visual Token Pruning Improve Calibration? The Role of Evidence Coverage in MLLMscited by 1
  11. 2026SCoOP: Semantic Consistent Opinion Pooling for Uncertainty Quantification in Multiple Vision-Language Model Systemscited by 0
  12. 2026Bridging the Missing-Modality Gap: Improving Text-Only Calibration of Vision Language Modelscited by 0
7 more
  1. 2026Uncertainty-Aware Decision Making in Multimodal Large Language Modelscited by 0
  2. 2026When Safety Overrides Vision: Exploring Dynamics between Vision Influence and Safety Alignment in Vision-Language Modelscited by 0
  3. 2026CUSP: Decomposable Collective Uncertainty for Multi-Agent Multimodal Reasoningcited by 0
  4. 2026Dissecting Training-Free Uncertainty Estimation in Multimodal Large Language Modelscited by 0
  5. 2026Audio LLMs Know When They Can't Hear Youcited by 0
  6. 2026Estimating Uncertainty of Omnimodal Large Language Models via External Omnimodal Embeddingscited by 0
  7. 2026Visual Dependence Does Not Imply Answerability: A Systematic Analysis of Answerability Signals in Frozen InstructBLIPcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios
  2. 2025Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs
  3. 2023SEED-Bench-2: Benchmarking Multimodal Large Language Models
  4. 2025Benchmarking Large and Small MLLMs
  5. 2025The Art of Saying "Maybe": A Conformal Lens for Uncertainty Benchmarking in VLMs
  6. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks
  7. 2024A Survey on Benchmarks of Multimodal Large Language Models
  8. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks