mlsanityData to 14 Oct 2026

Look Again Before You Abstain:Budgeted Conformal Evidence Acquisition for Reliable Vision-Language Model

arXiv 2606.16667 ↗First posted 15 Jun 2026cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large vision-language models (LVLMs) hallucinate: they assert visual details that the image does not support. A principled remedy is selective prediction with a distribution-free guarantee-verify each claim and abstain when the claim is not grounded, so that the hallucination rate among asserted claims is provably bounded. We show, however, that this guarantee is bought at a brutal price: to keep the hallucination rate below 5% on a balanced object-existence benchmark, a state-of-the-art conformal filter must abstain on more than 80% of claims. We argue that abstention is wasteful when more visual evidence is cheaply available, and introduce Budgeted Conformal Evidence Acquisition (BCEA), which replaces the binary answer/abstain decision with a three-way choice: answer, abstain, or acquire additional visual evidence by re-examining the image (zooming, cropping, or applying a claim-specific intervention) under a bounded

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 33 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

23 of 33 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding10 Jun
  2. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  3. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback1 Dec
  4. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  5. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  6. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov

Other references · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2025Qwen3-VL Technical Reportcited by 2,793
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2017Selective Classification for Deep Neural Networkscited by 852
  7. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  8. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 271
  9. 2020Estimating means of bounded random variables by bettingcited by 169
  10. 2022Game-theoretic statistics and safe anytime-valid inferencecited by 155
  11. 2024Language Models with Conformal Factuality Guaranteescited by 131
  12. 2024Selective "Selective Prediction": Reducing Unnecessary Abstention in Vision-Language Reasoningcited by 40
5 more
  1. 2025AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisitioncited by 21
  2. 2023Towards Perceiving Small Visual Details in Zero-shot Visual Question Answering with Multimodal LLMscited by 15
  3. 2025CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perceptioncited by 8
  4. 2024TUBench: Benchmarking Large Vision-Language Models on Trustworthiness with Unanswerable Questionscited by 7
  5. 2025Test-time augmentation improves efficiency in conformal predictioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Safe to Stop? Risk-Constrained Stopping for Sequential Clinical Diagnosis Agentscited by 1
  2. 2026Symmetry-Aware Likelihood-Orbit Aggregation for Selective Left-Right Claim Verificationcited by 0
  3. 2026Have I Seen Enough? Frozen Video-Language Models Encode Evidence Readinesscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026From Token Correction to Claim Admission: Training-Free Hallucination Mitigation for Large Vision-Language Models
  2. 2026CDH-Bench: A Commonsense-Driven Hallucination Benchmark for Evaluating Visual Fidelity in Vision-Language Models
  3. 2026ReWEIGH the Evidence: Calibrating Token-Level Ordinal Visual Evidence to Mitigate Hallucinations in Large Vision-Language Models
  4. 2026IntroConformal: Conformal Factuality Guarantees for Large Vision-Language Models via Introspective Signals
  5. 2025Towards Statistical Factuality Guarantee for Large Vision-Language Models
  6. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models
  7. 2026Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steering
  8. 2026Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation