mlsanityData to 7 Oct 2026

PROVE: Proof-guided Regime-aware Operator Verification for Hallucination Detection in Medical Visual Question Answering

arXiv 2609.29604 ↗First posted 28 Aug 2026cs.CV cs.MMcited by 0

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

In medical visual question answering (VQA), hallucinations of vision-language models (VLMs) may lead to confident but incorrect responses, raising the risk of diagnostic errors. Existing hallucination detection methods uniformly estimate the reliability of VLM outputs from response consistency or visual evidence. However, such uniform verification across questions ignores question-specific characteristics, resulting in missed overconfident errors and false alarms from over-verification. We present PROVE (Proof-guided Regime-aware Operator Verification), a black-box detector that adapts verification strategy to the evidential structure of each question. PROVE classifies questions into three verification regimes based on what kind of visual proof they demand, activates a regime-specific subset of five complementary operators, and adjusts operator importance per question through a lightweight calibration layer conditioned on deterministic question-answer features. PROVE uses question-specific evidence to reweight operators and produce a calibrated risk score. Evaluated on 8048 test samples across three medical VQA benchmarks and four frontier VLMs, PROVE achieves 0.821 AUROC, outperforming the strongest baseline by +0.159, with consistent gains across all models and benchmarks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 29 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

11 of 29 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Localizing Before Answering: A Hallucination Evaluation Benchmark for Grounded Medical Multimodal LLMs30 Apr
  2. 2024Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning15 Dec
  3. 2024Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification2 Jul

Other references · 8

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025Qwen3-VL Technical Reportcited by 2,534
  3. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  4. 2024GREEN: Generative Radiology Report Evaluation and Error Notationcited by 124
  5. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 89
  6. 2025MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Modelscited by 33
  7. 2025CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretationcited by 7
  8. 2025When to Trust the Answer: Question-Aligned Semantic Nearest Neighbor Entropy for Safer Surgical VQAcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026V-Loop: Visual Logical Loop Verification for Hallucination Detection in Medical Visual Question Answering
  2. 2026Deterministic Hallucination Detection in Medical VQA via Confidence-Evidence Bayesian Gain
  3. 2026VIHD: Visual Intervention-based Hallucination Detection for Medical Visual Question Answering
  4. 2026FaithSCAN: Model-Driven Single-Pass Hallucination Detection for Faithful Visual Question Answering
  5. 2025Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decoding
  6. 2024Towards a Systematic Evaluation of Hallucinations in Large-Vision Language Models
  7. 2025UniVRSE: Unified Vision-conditioned Response Semantic Entropy for Hallucination Detection in Medical Vision-Language Models
  8. 2025HEDGE: Hallucination Estimation via Dense Geometric Entropy for VQA with Vision-Language Models