# Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models

arXiv: https://arxiv.org/abs/2412.14660 · HTML: https://mlsanity.com/paper/2412.14660/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 19 Dec 2024.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 14 of 14)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Aligning Large Multimodal Models with Factually Augmented RLHF](https://mlsanity.com/paper/2309.14525/) (2023)
- [PathVQA: 30000+ Questions for Medical Visual Question Answering](https://mlsanity.com/paper/2003.10286/) (2020)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [Can AI Assistants Know What They Don't Know?](https://mlsanity.com/paper/2401.13275/) (2024)
- [Investigating Uncertainty Calibration of Aligned Language Models under the Multiple-Choice Setting](https://mlsanity.com/paper/2310.11732/) (2023)
- [Exploring Transferability of Multimodal Adversarial Samples for Vision-Language Pre-training Models with Contrastive Learning](https://mlsanity.com/paper/2308.12636/) (2023)
- [Deep sub-ensembles meets quantile regression: uncertainty-aware imputation for time series](https://mlsanity.com/paper/2312.01294/) (2023)

## Built on by (most cited first, 19 of 19)

- [Evaluating Uncertainty and Quality of Vision-Language-Action-enabled Robots](https://mlsanity.com/paper/2507.17049/) (2025)
- [Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Models](https://mlsanity.com/paper/2505.20236/) (2025)
- [Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volume](https://mlsanity.com/paper/2602.24195/) (2026)
- [Uncertainty-o: One Model-agnostic Framework for Unveiling Uncertainty in Large Multimodal Models](https://mlsanity.com/paper/2506.07575/) (2025)
- [Explicit Abstention Knobs for Predictable Reliability in Video Question Answering](https://mlsanity.com/paper/2601.00138/) (2025)
- [Don't Blink: Evidence Collapse during Multimodal Reasoning](https://mlsanity.com/paper/2604.04207/) (2026)
- [Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoning](https://mlsanity.com/paper/2507.10007/) (2025)
- [Analysis of Image-and-Text Uncertainty Propagation in Multimodal Large Language Models with Cardiac MR-Based Applications](https://mlsanity.com/paper/2507.12945/) (2025)
- [FaithSCAN: Model-Driven Single-Pass Hallucination Detection for Faithful Visual Question Answering](https://mlsanity.com/paper/2601.00269/) (2026)
- [When Does Visual Token Pruning Improve Calibration? The Role of Evidence Coverage in MLLMs](https://mlsanity.com/paper/2604.12035/) (2026)
- [SCoOP: Semantic Consistent Opinion Pooling for Uncertainty Quantification in Multiple Vision-Language Model Systems](https://mlsanity.com/paper/2603.23853/) (2026)
- [Bridging the Missing-Modality Gap: Improving Text-Only Calibration of Vision Language Models](https://mlsanity.com/paper/2605.12517/) (2026)
- [Uncertainty-Aware Decision Making in Multimodal Large Language Models](https://mlsanity.com/paper/2608.17084/) (2026)
- [When Safety Overrides Vision: Exploring Dynamics between Vision Influence and Safety Alignment in Vision-Language Models](https://mlsanity.com/paper/2608.18628/) (2026)
- [CUSP: Decomposable Collective Uncertainty for Multi-Agent Multimodal Reasoning](https://mlsanity.com/paper/2609.05708/) (2026)
- [Dissecting Training-Free Uncertainty Estimation in Multimodal Large Language Models](https://mlsanity.com/paper/2609.22206/) (2026)
- [Audio LLMs Know When They Can't Hear You](https://mlsanity.com/paper/2609.30625/) (2026)
- [Estimating Uncertainty of Omnimodal Large Language Models via External Omnimodal Embeddings](https://mlsanity.com/paper/2610.16933/) (2026)
- [Visual Dependence Does Not Imply Answerability: A Systematic Analysis of Answerability Signals in Frozen InstructBLIP](https://mlsanity.com/paper/2610.17902/) (2026)

## Abstract (arXiv)

Multimodal large language models (MLLMs) combine visual and textual data for tasks such as image captioning and visual question answering. Proper uncertainty calibration is crucial, yet challenging, for reliable use in areas like healthcare and autonomous driving. This paper investigates representative MLLMs, focusing on their calibration across various scenarios, including before and after visual fine-tuning, as well as before and after multimodal training of the base LLMs. We observed miscalibration in their performance, and at the same time, no significant differences in calibration across these scenarios. We also highlight how uncertainty differs between text and images and how their integration affects overall uncertainty. To better understand MLLMs' miscalibration and their ability to self-assess uncertainty, we construct the IDK (I don't know) dataset, which is key to evaluating how they handle unknowns. Our findings reveal that MLLMs tend to give answers rather than admit uncertainty, but this self-assessment improves with proper prompt adjustments. Finally, to calibrate MLLMs and enhance model reliability, we propose techniques such as temperature scaling and iterative prompt optimization. Our results provide insights into improving MLLMs for effective and responsible deployment in multimodal applications. Code and IDK dataset: https://github.com/hfutml/Calibration-MLLM.