mlsanityData to 7 Oct 2026

Uncertainty-Aware Evaluation for Vision-Language Models

arXiv 2402.14418 ↗First posted 22 Feb 2024cs.CV cs.AIcited by 28

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Vision-Language Models like GPT-4, LLaVA, and CogVLM have surged in popularity recently due to their impressive performance in several vision-language tasks. Current evaluation methods, however, overlook an essential component: uncertainty, which is crucial for a comprehensive assessment of VLMs. Addressing this oversight, we present a benchmark incorporating uncertainty quantification into evaluating VLMs. Our analysis spans 20+ VLMs, focusing on the multiple-choice Visual Question Answering (VQA) task. We examine models on 5 datasets that evaluate various vision-language capabilities. Using conformal prediction as an uncertainty estimation approach, we demonstrate that the models' uncertainty is not aligned with their accuracy. Specifically, we show that models with the highest accuracy may also have the highest uncertainty, which confirms the importance of measuring it for VLMs. Our empirical findings also reveal a correlation between model uncertainty and its language model part.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 64 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

53 of 64 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 52

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2017On Calibration of Modern Neural Networkscited by 4,860
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  11. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  12. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
38 more
  1. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  3. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  4. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  5. 2019Towards VQA Models That Can Readcited by 1,696
  6. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  8. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  9. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  10. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  11. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  12. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  13. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  14. 2016A Diagram Is Worth A Dozen Imagescited by 793
  15. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  16. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  17. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  18. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  19. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  20. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  21. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  22. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  23. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  24. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
  25. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  26. 2018nocaps: novel object captioning at scalecited by 444
  27. 2020Classification with Valid and Adaptive Coveragecited by 390
  28. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 356
  29. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 298
  30. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  31. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 277
  32. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 270
  33. 2023Conformal Language Modelingcited by 163
  34. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 131
  35. 2024Benchmarking LLMs via Uncertainty Quantificationcited by 124
  36. 2020Conformal Prediction: a Unified Review of Theory and New Challengescited by 118
  37. 2023CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generationcited by 63
  38. 2021OOD-CV: A Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Imagescited by 46

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 11; 2026: 12.

28 papers.

Sort:

By area:

  1. 2024Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratingscited by 44
  2. 2025Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Modelscited by 20
  3. 2024Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Modelscited by 18
  4. 2024ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videoscited by 18
  5. 2025Object Detection with Multimodal Large Vision-Language Models: An In-depth Reviewcited by 14
  6. 2025Seeing and Reasoning with Confidence: Supercharging Multimodal LLMs with an Uncertainty-Aware Agentic Frameworkcited by 12
  7. 2025Calibrating Uncertainty Quantification of Multi-Modal LLMs using Groundingcited by 8
  8. 2026The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Modelscited by 5
  9. 2026Geometry-Calibrated Conformal Abstention for Language Modelscited by 5
  10. 2026Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Modelscited by 4
  11. 2024Improving Medical Diagnostics with Vision-Language Models: Convex Hull-Based Uncertainty Analysiscited by 3
  12. 2025Data-Driven Calibration of Prediction Sets in Large Vision-Language Models Based on Inductive Conformal Predictioncited by 3
16 more
  1. 2025MTRE: Multi-Token Reliability Estimation for Hallucination Detection in VLMscited by 3
  2. 2025PARC: A Quantitative Framework Uncovering the Symmetries within Vision Language Modelscited by 3
  3. 2026Set-Valued Prediction for Large Language Models with Feasibility-Aware Coverage Guaranteescited by 3
  4. 2024Quality Control for Radiology Report Generation Models via Auxiliary Auditing Componentscited by 2
  5. 2025Are vision language models robust to uncertain inputs?cited by 2
  6. 2025Prune-Then-Plan: Step-Level Calibration for Stable Frontier Exploration in Embodied Question Answeringcited by 2
  7. 2026Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasetscited by 2
  8. 2025Analysis of Image-and-Text Uncertainty Propagation in Multimodal Large Language Models with Cardiac MR-Based Applicationscited by 1
  9. 2025Conformal Sets in Multiple-Choice Question Answering under Black-Box Settings with Provable Coverage Guaranteescited by 1
  10. 2026Assessing Vision-Language Models for Perception in Autonomous Underwater Robotic Softwarecited by 0
  11. 2026DANCE: Doubly Adaptive Neighborhood Conformal Estimationcited by 0
  12. 2026Logit-Level Uncertainty Quantification in Vision-Language Models for Histopathology Image Analysiscited by 0
  13. 2026Reasoning Portability: Guiding Continual Learning for MLLMs in the RLVR Eracited by 0
  14. 2026VISTA: Auditing Semantic Divergence in Vision-Language Modelscited by 0
  15. 2026Uncertainty-Aware Decision Making in Multimodal Large Language Modelscited by 0
  16. 2026Can VLMs Reliably Assess Sidewalk Accessibility Attributes from Pedestrian-Level Imagery?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Art of Saying "Maybe": A Conformal Lens for Uncertainty Benchmarking in VLMs
  2. 2026Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation
  3. 2026VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluation
  4. 2026VLM-UQBench: A Benchmark for Modality-Specific and Cross-Modality Uncertainties in Vision Language Models
  5. 2026FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty
  6. 2024Overconfidence is Key: Verbalized Uncertainty Evaluation in Large Language and Vision-Language Models
  7. 2026Seeing Isn't Believing: Uncovering Blind Spots in Evaluator Vision-Language Models
  8. 2024Post-hoc Probabilistic Vision-Language Models