# Uncertainty-Aware Evaluation for Vision-Language Models

arXiv: https://arxiv.org/abs/2402.14418 · HTML: https://mlsanity.com/paper/2402.14418/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 53 of its 64 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Hallucination mitigation in LVLMs](https://mlsanity.com/mllm/hallucination-mitigation-in-lvlms/)

## Builds on (most cited first, 30 of 53)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension](https://mlsanity.com/paper/2307.16125/) (2023)
- [A Diagram Is Worth A Dozen Images](https://mlsanity.com/paper/1603.07396/) (2016)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [CogVLM: Visual Expert for Pretrained Language Models](https://mlsanity.com/paper/2311.03079/) (2023)
- [EVA: Exploring the Limits of Masked Visual Representation Learning at Scale](https://mlsanity.com/paper/2211.07636/) (2022)

## Built on by (most cited first, 29 of 29)

- [Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings](https://mlsanity.com/paper/2404.16820/) (2024)
- [Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models](https://mlsanity.com/paper/2502.06884/) (2025)
- [Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Models](https://mlsanity.com/paper/2410.08174/) (2024)
- [ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videos](https://mlsanity.com/paper/2411.14901/) (2024)
- [Object Detection with Multimodal Large Vision-Language Models: An In-depth Review](https://mlsanity.com/paper/2508.19294/) (2025)
- [Seeing and Reasoning with Confidence: Supercharging Multimodal LLMs with an Uncertainty-Aware Agentic Framework](https://mlsanity.com/paper/2503.08308/) (2025)
- [Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding](https://mlsanity.com/paper/2505.03788/) (2025)
- [The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Models](https://mlsanity.com/paper/2603.16728/) (2026)
- [Geometry-Calibrated Conformal Abstention for Language Models](https://mlsanity.com/paper/2604.27914/) (2026)
- [Set-Valued Prediction for Large Language Models with Feasibility-Aware Coverage Guarantees](https://mlsanity.com/paper/2603.22966/) (2026)
- [Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Models](https://mlsanity.com/paper/2604.25591/) (2026)
- [Improving Medical Diagnostics with Vision-Language Models: Convex Hull-Based Uncertainty Analysis](https://mlsanity.com/paper/2412.00056/) (2024)
- [Data-Driven Calibration of Prediction Sets in Large Vision-Language Models Based on Inductive Conformal Prediction](https://mlsanity.com/paper/2504.17671/) (2025)
- [MTRE: Multi-Token Reliability Estimation for Hallucination Detection in VLMs](https://mlsanity.com/paper/2505.11741/) (2025)
- [Are vision language models robust to uncertain inputs?](https://mlsanity.com/paper/2505.11804/) (2025)
- [PARC: A Quantitative Framework Uncovering the Symmetries within Vision Language Models](https://mlsanity.com/paper/2506.14808/) (2025)
- [Quality Control for Radiology Report Generation Models via Auxiliary Auditing Components](https://mlsanity.com/paper/2407.21638/) (2024)
- [Conformal Sets in Multiple-Choice Question Answering under Black-Box Settings with Provable Coverage Guarantees](https://mlsanity.com/paper/2508.05544/) (2025)
- [Prune-Then-Plan: Step-Level Calibration for Stable Frontier Exploration in Embodied Question Answering](https://mlsanity.com/paper/2511.19768/) (2025)
- [Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets](https://mlsanity.com/paper/2606.25760/) (2026)
- [Analysis of Image-and-Text Uncertainty Propagation in Multimodal Large Language Models with Cardiac MR-Based Applications](https://mlsanity.com/paper/2507.12945/) (2025)
- [Assessing Vision-Language Models for Perception in Autonomous Underwater Robotic Software](https://mlsanity.com/paper/2602.10655/) (2026)
- [DANCE: Doubly Adaptive Neighborhood Conformal Estimation](https://mlsanity.com/paper/2602.20652/) (2026)
- [Logit-Level Uncertainty Quantification in Vision-Language Models for Histopathology Image Analysis](https://mlsanity.com/paper/2603.03527/) (2026)
- [Reasoning Portability: Guiding Continual Learning for MLLMs in the RLVR Era](https://mlsanity.com/paper/2605.18903/) (2026)
- [VISTA: Auditing Semantic Divergence in Vision-Language Models](https://mlsanity.com/paper/2607.02995/) (2026)
- [Uncertainty-Aware Decision Making in Multimodal Large Language Models](https://mlsanity.com/paper/2608.17084/) (2026)
- [Can VLMs Reliably Assess Sidewalk Accessibility Attributes from Pedestrian-Level Imagery?](https://mlsanity.com/paper/2609.17882/) (2026)
- [How Well Do LLMs Reason with Noisy Evidence? An Active Visual Reasoning Benchmark](https://mlsanity.com/paper/2610.07751/) (2026)

## Abstract (arXiv)

Vision-Language Models like GPT-4, LLaVA, and CogVLM have surged in popularity recently due to their impressive performance in several vision-language tasks. Current evaluation methods, however, overlook an essential component: uncertainty, which is crucial for a comprehensive assessment of VLMs. Addressing this oversight, we present a benchmark incorporating uncertainty quantification into evaluating VLMs. Our analysis spans 20+ VLMs, focusing on the multiple-choice Visual Question Answering (VQA) task. We examine models on 5 datasets that evaluate various vision-language capabilities. Using conformal prediction as an uncertainty estimation approach, we demonstrate that the models' uncertainty is not aligned with their accuracy. Specifically, we show that models with the highest accuracy may also have the highest uncertainty, which confirms the importance of measuring it for VLMs. Our empirical findings also reveal a correlation between model uncertainty and its language model part.