mlsanityData to 7 Oct 2026

VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluation

arXiv 2602.21054 ↗First posted 24 Feb 2026cs.CV cs.AI cs.CLcited by 5

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large Vision-Language Models (LVLMs) frequently hallucinate, limiting their safe deployment in real-world applications. Existing LLM self-evaluation methods rely on a model's ability to estimate the correctness of its own outputs, which can improve deployment reliability; however, they depend heavily on language priors and are therefore ill-suited for evaluating vision-conditioned predictions. We propose VAUQ, a vision-aware uncertainty quantification framework for LVLM self-evaluation that explicitly measures how strongly a model's output depends on visual evidence. VAUQ introduces the Image-Information Score (IS), which captures the reduction in predictive uncertainty attributable to visual input, and an unsupervised core-region masking strategy that amplifies the influence of salient regions. Combining predictive entropy with this core-masked IS yields a training-free scoring function that reliably reflects answer correctness. Comprehensive experiments show that VAUQ consistently outperforms existing self-evaluation methods across multiple datasets.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 52 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

34 of 52 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Mitigating Hallucinations in Large Vision-Language Models by Self-Injecting Hallucinations14 Sep
  2. 2025GLSim: Detecting Object Hallucinations in LVLMs via Global-Local Similarity27 Aug
  3. 2025TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination Via Latent Truthful-Guided Pre-Intervention13 Mar
  4. 2024Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection18 Dec
  5. 2024Beyond Logit Lens: Contextual Embeddings for Robust Hallucination Detection & Grounding in VLMs28 Nov
  6. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens23 Nov
  7. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimation18 Nov
  8. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  9. 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention18 Jun
  10. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  11. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun

Other references · 23

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  6. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  8. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  9. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  10. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  11. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  12. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
11 more
  1. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  2. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  3. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  4. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 193
  5. 2024Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluationcited by 123
  6. 2019Improving Referring Expression Grounding with Cross-modal Attention-guided Erasingcited by 88
  7. 2024Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluationcited by 56
  8. 2025Steer LLM Latents for Hallucination Detectioncited by 49
  9. 2023Mitigating Spurious Correlations in Multi-modal Models during Fine-tuningcited by 47
  10. 2024Probing Visual Language Priors in VLMscited by 44
  11. 2025Semantic Volume: Quantifying and Detecting both External and Internal Uncertainty in LLMscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoringcited by 6
  2. 2026When Does Consensus Mean Correctness? Measuring the Agreement-Accuracy Coupling with Semantics-Preserving Re-Renderingcited by 0
  3. 2026LookBack: Where and How to Score LVLM Responses via Visual Reference Usagecited by 0
  4. 2026Uncertainty-Aware Decision Making in Multimodal Large Language Modelscited by 0
  5. 2026Broadening Uncertainty Estimation for Audio Question Answering Across Methods, Formats, and Inputscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation
  2. 2024Uncertainty-Aware Evaluation for Vision-Language Models
  3. 2025The Art of Saying "Maybe": A Conformal Lens for Uncertainty Benchmarking in VLMs
  4. 2026VLM-UQBench: A Benchmark for Modality-Specific and Cross-Modality Uncertainties in Vision Language Models
  5. 2026Dynamic Distribution-Aware Uncertainty Tracking in Vision-Language Representation Learning
  6. 2025MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMs
  7. 2026FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty
  8. 2026VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning