mlsanityData to 7 Oct 2026

Localizing Before Answering: A Hallucination Evaluation Benchmark for Grounded Medical Multimodal LLMs

arXiv 2505.00744 ↗First posted 30 Apr 2025cs.CVcited by 14

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Medical Large Multi-modal Models (LMMs) have demonstrated remarkable capabilities in medical data interpretation. However, these models frequently generate hallucinations contradicting source evidence, particularly due to inadequate localization reasoning. This work reveals a critical limitation in current medical LMMs: instead of analyzing relevant pathological regions, they often rely on linguistic patterns or attend to irrelevant image areas when responding to disease-related queries. To address this, we introduce HEAL-MedVQA (Hallucination Evaluation via Localization MedVQA), a comprehensive benchmark designed to evaluate LMMs' localization abilities and hallucination robustness. HEAL-MedVQA features (i) two innovative evaluation protocols to assess visual and textual shortcut learning, and (ii) a dataset of 67K VQA pairs, with doctor-annotated anatomical segmentation masks for pathological regions. To improve visual reasoning, we propose the Localize-before-Answer (LobA) framework, which trains LMMs to localize target regions of interest and self-prompt to emphasize segmented pathological areas, generating grounded and reliable answers. Experimental results demonstrate that our approach significantly outperforms state-of-the-art biomedical LMMs on the challenging HEAL-MedVQA benchmark, advancing robustness in medical VQA.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 49 references that are in the map. First posted on arXiv 30 Apr 2025.

Builds on

34 of 49 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Multi-Modal Hallucination Control by Visual Information Grounding20 Mar
  2. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  3. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  4. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun

Other references · 30

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Segment Anythingcited by 8,158
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  10. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  11. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  12. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
18 more
  1. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  2. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  3. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  4. 2020VinDr-CXR: An open dataset of chest X-rays with radiologist's annotationscited by 236
  5. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 182
  6. 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 176
  7. 2023MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiologycited by 149
  8. 2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Modelscited by 64
  9. 2024Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Trainingcited by 62
  10. 2023MedBLIP: Bootstrapping Language-Image Pre-training from 3D Medical Images and Textscited by 51
  11. 2023VCoder: Versatile Vision Encoders for Multimodal Large Language Modelscited by 42
  12. 2023Prompt Highlighter: Interactive Control for Multi-Modal LLMscited by 29
  13. 2022Detailed Annotations of Chest X-Rays via CT Projection for Report Understandingcited by 24
  14. 2024Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQAcited by 19
  15. 2024Hallucination Benchmark in Medical Visual Question Answeringcited by 18
  16. 2024Incorporating Visual Experts to Resolve the Information Loss in Multimodal Large Language Modelscited by 12
  17. 2023Localized Questions in Medical Visual Question Answeringcited by 5
  18. 2024Targeted Visual Prompting for Medical Visual Question Answeringcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 9.

14 papers.

Sort:

By area:

  1. 2025Instruction-Guided Lesion Segmentation for Chest X-rays with Automatically Generated Large-Scale Datasetcited by 5
  2. 2025Seeing the Trees for the Forest: Rethinking Weakly-Supervised Medical Visual Groundingcited by 3
  3. 2026CXReasonAgent: Evidence-Grounded Diagnostic Reasoning Agent for Chest X-rayscited by 3
  4. 2025Foundation Models in Biomedical Imaging: Turning Hype into Realitycited by 2
  5. 2025CoTBox-TTT: Grounding Medical VQA with Visual Chain-of-Thought Boxes During Test-time Trainingcited by 1
  6. 2026Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidencecited by 1
  7. 2026SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessitycited by 1
  8. 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomycited by 1
  9. 2025Colon-X: Advancing Intelligent Colonoscopy toward Clinical Reasoningcited by 0
  10. 2026MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidencecited by 0
  11. 2026Rad-VLSM: A Cross-Modal Framework with Semantics-Assisted Prompting for Medical Segmentation and Diagnosiscited by 0
  12. 2026SHOVIR: A Benchmark for Evaluating Vision Shortcut Learning in Radiology Report Generationcited by 0
2 more
  1. 2026PROVE: Proof-guided Regime-aware Operator Verification for Hallucination Detection in Medical Visual Question Answeringcited by 0
  2. 2026CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Detecting and Evaluating Medical Hallucinations in Large Vision Language Models
  2. 2024MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context
  3. 2025MedHallTune: An Instruction-Tuning Benchmark for Mitigating Medical Hallucination in Vision-Language Models
  4. 2026Deterministic Hallucination Detection in Medical VQA via Confidence-Evidence Bayesian Gain
  5. 2026Med-StepBench: A Hierarchical Reasoning Framework for Evaluating Hallucinations in Medical Vision-Language Models
  6. 2025MedHEval: Benchmarking Hallucinations and Mitigation Strategies in Medical Large Vision-Language Models
  7. 2026Med-HEAL: Analyzing and Mitigating Hallucinations in Medical LLMs with Hallucination-Aware In-Context Learning
  8. 2024MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models