mlsanityData to 7 Oct 2026

MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models

arXiv 2409.15477 ↗First posted 23 Sep 2024cs.CVcited by 22

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have tremendous potential to improve the accuracy, availability, and cost-effectiveness of healthcare by providing automated solutions or serving as aids to medical professionals. Despite promising first steps in developing medical MLLMs in the past few years, their capabilities and limitations are not well-understood. Recently, many benchmark datasets have been proposed that test the general medical knowledge of such models across a variety of medical areas. However, the systematic failure modes and vulnerabilities of such models are severely underexplored with most medical benchmarks failing to expose the shortcomings of existing models in this safety-critical domain. In this paper, we introduce MediConfusion, a challenging medical Visual Question Answering (VQA) benchmark dataset, that probes the failure modes of medical MLLMs from a vision perspective. We reveal that state-of-the-art models are easily confused by image pairs that are otherwise visually dissimilar and clearly distinct for medical experts. Strikingly, all available models (open-source or proprietary) achieve performance below random guessing on MediConfusion, raising serious concerns about the reliability of existing medical MLLMs for healthcare deployment. We also extract common patterns of model failure that may help the design of a new generation of more trustworthy and reliable MLLMs in healthcare.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 42 of its 51 references that are in the map. First posted on arXiv 23 Sep 2024.

Builds on

42 of 51 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Beyond the Hype: A dispassionate look at vision-language models in medical scenario16 Aug
  2. 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI6 Aug
  3. 2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models10 Jun
  4. 2024A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models17 Feb
  5. 2024MultiMedEval: A Benchmark and a Toolkit for Evaluating Medical Vision-Language Models14 Feb
  6. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM14 Feb
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  10. 2020Learning to summarize from human feedbackcited by 2,562
  11. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  12. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
23 more
  1. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  2. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  3. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  4. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  5. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
  6. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  7. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  8. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  9. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  10. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  11. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  12. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  13. 2023Detecting and Preventing Hallucinations in Large Vision Language Modelscited by 254
  14. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 245
  15. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  16. 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
  17. 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 176
  18. 2023ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Promptscited by 160
  19. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 88
  20. 2023Can GPT-4V(ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosiscited by 77
  21. 2024Asymptotics of Language Model Alignmentcited by 46
  22. 2023VCoder: Versatile Vision Encoders for Multimodal Large Language Modelscited by 42
  23. 2023Mass-Producing Failures of Multimodal Systems with Language Modelscited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 14; 2026: 7.

22 papers.

Sort:

By area:

  1. 2025UniEdit-Flow: Unleashing Inversion and Editing in the Era of Flow Modelscited by 38
  2. 2025R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Sciencecited by 33
  3. 2025EndoBench: A Comprehensive Evaluation of Multi-Modal Large Language Models for Endoscopy Analysiscited by 13
  4. 20243D Shape Tokenization via Latent Flow Matchingcited by 10
  5. 2025MedLA: A Logic-Driven Multi-Agent Framework for Complex Medical Reasoning with Large Language Modelscited by 9
  6. 2025VGDFR: Diffusion-based Video Generation with Dynamic Latent Frame Ratecited by 7
  7. 2025PET2Rep: Towards Vision-Language Model-Drived Automated Radiology Report Generation for Positron Emission Tomographycited by 6
  8. 2025Hyperphantasia: A Benchmark for Evaluating the Mental Visualization Capabilities of Multimodal LLMscited by 5
  9. 2025AstroMMBench: A Benchmark for Evaluating Multimodal Large Language Models Capabilities in Astronomycited by 3
  10. 2025HARMONY: Hidden Activation Representations and Model Output-Aware Uncertainty Estimation for Vision-Language Modelscited by 3
  11. 2025Domain Specific Benchmarks for Evaluating Multimodal Large Language Modelscited by 2
  12. 2026Vision-language models for chest radiography do not always need the imagecited by 2
10 more
  1. 2025MedBLINK: Probing Basic Perception in Multimodal Language Models for Medicinecited by 1
  2. 2025Multi-human Interactive Talking Datasetcited by 1
  3. 2026Benchmarking Egocentric Clinical Intent Understanding Capability for Medical Multimodal Large Language Modelscited by 1
  4. 2026AMVICC: A Novel Benchmark for Cross-Modal Failure Mode Profiling for VLMs and IGMscited by 1
  5. 2025MultiMedEdit: A Scenario-Aware Benchmark for Evaluating Knowledge Editing in Medical VQAcited by 0
  6. 2025From Events to Clarity: The Event-Guided Diffusion Framework for Dehazingcited by 0
  7. 2026DoubleTake: Contrastive Reasoning for Faithful Decision-Making in Medical Imagingcited by 0
  8. 2026Do Medical Vision Models Reason About Anatomy? Probing the Spatial Inductive Biases of Learned Visual Representationscited by 0
  9. 2026Beyond Benchmark Scores: Auditing Medical Vision-Language Models for Chest X-Ray Tuberculosis Screeningcited by 0
  10. 2026Applying Language Models in Clinical Medicine: Recent Trends and Perspectivescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence
  2. 2024MedThink: Explaining Medical Visual Question Answering via Multimodal Decision-Making Rationale
  3. 2024Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQA
  4. 2024PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imaging
  5. 2025Medical Large Vision Language Models with Multi-Image Visual Ability
  6. 2025Fleming-VL: Towards Universal Medical Visual Reasoning with Multimodal LLMs
  7. 2026A visual large language foundational model for medical image recognition using clinician-contributed online resources
  8. 2025MedErr-CT: A Visual Question Answering Benchmark for Identifying and Correcting Errors in CT Reports