mlsanityData to 7 Oct 2026

Detecting and Evaluating Medical Hallucinations in Large Vision Language Models

arXiv 2406.10185 ↗First posted 14 Jun 2024cs.CVcited by 57

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large Vision Language Models (LVLMs) are increasingly integral to healthcare applications, including medical visual question answering and imaging report generation. While these models inherit the robust capabilities of foundational Large Language Models (LLMs), they also inherit susceptibility to hallucinations-a significant concern in high-stakes medical contexts where the margin for error is minimal. However, currently, there are no dedicated methods or benchmarks for hallucination detection and evaluation in the medical field. To bridge this gap, we introduce Med-HallMark, the first benchmark specifically designed for hallucination detection and evaluation within the medical multimodal domain. This benchmark provides multi-tasking hallucination support, multifaceted hallucination data, and hierarchical hallucination categorization. Furthermore, we propose the MediHall Score, a new medical evaluative metric designed to assess LVLMs' hallucinations through a hierarchical scoring system that considers the severity and type of hallucination, thereby enabling a granular assessment of potential clinical impacts. We also present MediHallDetector, a novel Medical LVLM engineered for precise hallucination detection, which employs multitask training for hallucination detection. Through extensive experimental evaluations, we establish baselines for popular LVLMs using our benchmark. The findings indicate that MediHall Score provides a more nuanced understanding of hallucination impacts compared to traditional metrics and demonstrate the enhanced performance of MediHallDetector. We hope this work can significantly improve the reliability of LVLMs in medical applications. All resources of this work have been released at https://github.com/ydk122024/Med-HallMark.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 29 references that are in the map. First posted on arXiv 14 Jun 2024.

Builds on

20 of 29 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
  2. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback22 Apr
  3. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
  4. 2024Unified Hallucination Detection for Multimodal Large Language Models5 Feb
  5. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
  6. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  7. 2023Evaluation and Analysis of Hallucination in Large Vision-Language Models29 Aug
  8. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  9. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  10. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 10

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  5. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  6. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  7. 2018Object Hallucination in Image Captioningcited by 661
  8. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  9. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  10. 2024MISS: A Generative Pretraining and Finetuning Approach for Med-VQAcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 23; 2026: 23.

57 papers.

Sort:

By area:

  1. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 84
  2. 2024VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledgecited by 50
  3. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Modelscited by 44
  4. 2024GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosiscited by 28
  5. 2025A Survey of Multimodal Hallucination Evaluation and Detectioncited by 26
  6. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  7. 2024CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generationcited by 25
  8. 2024MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimizationcited by 21
  9. 2024Has Multimodal Learning Delivered Universal Intelligence in Healthcare? A Comprehensive Surveycited by 20
  10. 2024MedHalu: Hallucinations in Responses to Healthcare Queries by Large Language Modelscited by 20
  11. 2024A Survey on Trustworthiness in Foundation Models for Medical Image Analysiscited by 16
  12. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Modelscited by 15
38 more
  1. 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolutioncited by 12
  2. 2024Towards Context-Aware Emotion Recognition Debiasing from a Causal Demystification Perspective via De-confounded Trainingcited by 11
  3. 2024Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representationscited by 10
  4. 2025Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Modelscited by 9
  5. 2025Mitigating Image Captioning Hallucinations in Vision-Language Modelscited by 7
  6. 2025MedHallTune: An Instruction-Tuning Benchmark for Mitigating Medical Hallucination in Vision-Language Modelscited by 6
  7. 2025Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decodingcited by 6
  8. 2025MindWatcher: Toward Smarter Multimodal Tool-Integrated Reasoningcited by 6
  9. 2025UniVRSE: Unified Vision-conditioned Response Semantic Entropy for Hallucination Detection in Medical Vision-Language Modelscited by 5
  10. 2025From Gaze to Insight: Bridging Human Visual Attention and Vision Language Model Explanation for Weakly-Supervised Medical Image Segmentationcited by 5
  11. 2025TECP: Token-Entropy Conformal Prediction for LLMscited by 5
  12. 2025Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generationcited by 5
  13. 2024Scorecards for Synthetic Medical Data Evaluation and Reportingcited by 4
  14. 2025CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decodingcited by 4
  15. 2026OPGAgent: An Agent for Auditable Dental Panoramic X-ray Interpretationcited by 4
  16. 2025Fair Foundation Models for Medical Image Analysis: Challenges and Perspectivescited by 3
  17. 2025A Scoping Review of Natural Language Processing in Addressing Medically Inaccurate Information: Errors, Misinformation, and Hallucinationcited by 2
  18. 2025Preemptive Hallucination Reduction: An Input-Level Approach for Multimodal Language Modelcited by 2
  19. 2025EH-Benchmark Ophthalmic Hallucination Benchmark and Agent-Driven Top-Down Traceable Reasoning Workflowcited by 2
  20. 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2
  21. 2026HERO: Hierarchical Evidential Reasoning Optimization for Radiology Report Generation via Reason-then-Summarizecited by 2
  22. 2026To Agree or To Be Right? The Grounding-Sycophancy Tradeoff in Medical Vision-Language Modelscited by 2
  23. 2026MedObvious: Exposing the Medical Moravec's Paradox in VLMs via Clinical Triagecited by 2
  24. 2025Meta-Entity Driven Triplet Mining for Aligning Medical Vision-Language Modelscited by 1
  25. 2026From Attribution to Abstention: Training-Free Attention-Based Auditing for Clinical Summarizationcited by 1
  26. 2026ClinCoT: Clinical-Aware Visual Chain-of-Thought for Medical Vision Language Modelscited by 1
  27. 2026Evaluating the Search Agent in a Parallel Worldcited by 1
  28. 2026Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectificationcited by 1
  29. 2026ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoningcited by 1
  30. 2026Hallucination Detection and Correction in Medical VLMs via Counter-Evidence Verificationcited by 1
  31. 2026Unified Hallucination Fuzzing for Multimodal Large Language Modelscited by 1
  32. 2025Curing Semantic Drift: A Dynamic Approach to Grounding Generation in Large Vision-Language Modelscited by 0
  33. 2025RadEyeVideo: Enhancing general-domain Large Vision Language Model for chest X-ray analysis with video representations of eye gazecited by 0
  34. 2025Hallucinations in medical devicescited by 0
  35. 2026Making medical vision-language models think causally across modalities with retrieval-augmented cross-modal reasoningcited by 0
  36. 2026Retina-RAG: Retrieval-Augmented Vision-Language Modeling for Joint Retinal Diagnosis and Clinical Report Generationcited by 0
  37. 2026MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidencecited by 0
  38. 2026HalluCXR: Benchmarking and Mitigating Hallucinations in Medical Vision-Language Models for Chest Radiograph Interpretationcited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context
  2. 2025MedHEval: Benchmarking Hallucinations and Mitigation Strategies in Medical Large Vision-Language Models
  3. 2024MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models
  4. 2026Med-StepBench: A Hierarchical Reasoning Framework for Evaluating Hallucinations in Medical Vision-Language Models
  5. 2026Med-HEAL: Analyzing and Mitigating Hallucinations in Medical LLMs with Hallucination-Aware In-Context Learning
  6. 2026Deterministic Hallucination Detection in Medical VQA via Confidence-Evidence Bayesian Gain
  7. 2025Localizing Before Answering: A Hallucination Evaluation Benchmark for Grounded Medical Multimodal LLMs
  8. 2026HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models