mlsanityData to 7 Oct 2026

CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models

arXiv 2406.06007 ↗First posted 10 Jun 2024cs.LG cs.CL cs.CVcited by 64

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Artificial intelligence has significantly impacted medical applications, particularly with the advent of Medical Large Vision Language Models (Med-LVLMs), sparking optimism for the future of automated and personalized healthcare. However, the trustworthiness of Med-LVLMs remains unverified, posing significant risks for future model deployment. In this paper, we introduce CARES and aim to comprehensively evaluate the Trustworthiness of Med-LVLMs across the medical domain. We assess the trustworthiness of Med-LVLMs across five dimensions, including trustfulness, fairness, safety, privacy, and robustness. CARES comprises about 41K question-answer pairs in both closed and open-ended formats, covering 16 medical image modalities and 27 anatomical regions. Our analysis reveals that the models consistently exhibit concerns regarding trustworthiness, often displaying factual inaccuracies and failing to maintain fairness across different demographic groups. Furthermore, they are vulnerable to attacks and demonstrate a lack of privacy awareness. We publicly release our benchmark and code in https://cares-ai.github.io/.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 58 of its 99 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

58 of 99 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models17 Feb
  2. 2024MultiMedEval: A Benchmark and a Toolkit for Evaluating Medical Vision-Language Models14 Feb
  3. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM14 Feb
  4. 2023OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialogue21 Jun
  5. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 53

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  7. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  8. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  9. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  10. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  11. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  12. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
38 more
  1. 2024Yi: Open Foundation Models by 01.AIcited by 706
  2. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  3. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
  4. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  5. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  6. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  7. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  8. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  9. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 314
  10. 2023Towards Generalist Biomedical AIcited by 286
  11. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  12. 2023Detecting and Preventing Hallucinations in Large Vision Language Modelscited by 254
  13. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  14. 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
  15. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  16. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 197
  17. 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 176
  18. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decodingcited by 171
  19. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  20. 2023Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challengescited by 131
  21. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performancecited by 111
  22. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 103
  23. 2017Deceiving Google's Perspective API Built for Detecting Toxic Commentscited by 99
  24. 2024Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequencescited by 93
  25. 2024Towards Conversational Diagnostic AIcited by 79
  26. 2023Can GPT-4V(ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosiscited by 77
  27. 2024Calibrated Self-Rewarding Vision Language Modelscited by 77
  28. 2024Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvementcited by 74
  29. 2024FairCLIP: Harnessing Fairness in Vision-Language Learningcited by 69
  30. 2024CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding Benchmarkcited by 54
  31. 2023A Systematic Evaluation of GPT-4V's Multimodal Capability for Medical Image Analysiscited by 37
  32. 2018The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesionscited by 26
  33. 2024Lumen: Unleashing Versatile Vision-Centric Capabilities of Large Multimodal Modelscited by 24
  34. 2023HGCLIP: Exploring Vision-Language Models with Graph Representations for Hierarchical Understandingcited by 21
  35. 2023NurViD: A Large Expert-Level Video Database for Nursing Procedure Activity Understandingcited by 20
  36. 2023A Foundational Multimodal Vision Language AI Assistant for Human Pathologycited by 19
  37. 2022Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Changecited by 18
  38. 2024From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalitiescited by 18

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 34; 2026: 17.

64 papers.

Sort:

By area:

  1. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scalecited by 194
  2. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 84
  3. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  4. 2025MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understandingcited by 66
  5. 2025QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Trainingcited by 38
  6. 2025MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoningcited by 37
  7. 2024AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Drivingcited by 35
  8. 2024MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Modelscited by 32
  9. 2024MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation modelscited by 22
  10. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipescited by 22
  11. 2024MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimizationcited by 21
  12. 2024Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoEcited by 18
38 more
  1. 2024A Survey on Trustworthiness in Foundation Models for Medical Image Analysiscited by 16
  2. 2025Localizing Before Answering: A Hallucination Evaluation Benchmark for Grounded Medical Multimodal LLMscited by 14
  3. 2024TP-DRSeg: Improving Diabetic Retinopathy Lesion Segmentation with Explicit Text-Prompts Assisted SAMcited by 13
  4. 2025CLIMB: Data Foundations for Large Scale Multimodal Clinical Foundation Modelscited by 13
  5. 2025EndoBench: A Comprehensive Evaluation of Multi-Modal Large Language Models for Endoscopy Analysiscited by 13
  6. 2025Exploring Typographic Visual Prompts Injection Threats in Cross-Modality Generation Modelscited by 12
  7. 2024MVTamperBench: Evaluating Robustness of Vision-Language Modelscited by 11
  8. 2024Beyond the Hype: A dispassionate look at vision-language models in medical scenariocited by 10
  9. 2025Anyprefer: An Agentic Framework for Preference Data Synthesiscited by 10
  10. 2025A Multimodal Multi-Agent Framework for Radiology Report Generationcited by 8
  11. 2025CityLens: Evaluating Large Vision-Language Models for Urban Socioeconomic Sensingcited by 7
  12. 2025CLINIC: Evaluating Multilingual Trustworthiness in Language Models for Healthcarecited by 7
  13. 2025MedHallTune: An Instruction-Tuning Benchmark for Mitigating Medical Hallucination in Vision-Language Modelscited by 6
  14. 2025Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decodingcited by 6
  15. 2025MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answeringcited by 5
  16. 2025CBVLM: Training-free Explainable Concept-based Large Vision Language Models for Medical Image Classificationcited by 4
  17. 2025Uncertainty-Driven Expert Control: Enhancing the Reliability of Medical Vision-Language Modelscited by 4
  18. 2025X-Ray-CoT: Interpretable Chest X-ray Diagnosis with Vision-Language Models via Chain-of-Thought Reasoningcited by 4
  19. 2025RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answeringcited by 4
  20. 2025Fair Foundation Models for Medical Image Analysis: Challenges and Perspectivescited by 3
  21. 2025Focus on What Matters: Enhancing Medical Vision-Language Models with Automatic Attention Alignment Tuningcited by 3
  22. 2025PAC Bench: Do Foundation Models Understand Prerequisites for Executing Manipulation Policies?cited by 3
  23. 2026PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifactscited by 3
  24. 2024Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Modelscited by 2
  25. 2025HSENet: Hybrid Spatial Encoding Network for 3D Medical Vision-Language Understandingcited by 1
  26. 2025VSF-Med:A Vulnerability Scoring Framework for Medical Vision-Language Modelscited by 1
  27. 2025A Multi-Agent System for Complex Reasoning in Radiology Visual Question Answeringcited by 1
  28. 2025MedBLINK: Probing Basic Perception in Multimodal Language Models for Medicinecited by 1
  29. 2025Med-VRAgent: A Framework for Medical Visual Reasoning-Enhanced Agentscited by 1
  30. 2026Benchmarking Bias Mitigation Toward Fairness Without Harm from Vision to LVLMscited by 1
  31. 2026Consistent but Dangerous: Per-Sample Safety Classification Reveals False Reliability in Medical Vision-Language Modelscited by 1
  32. 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomycited by 1
  33. 2026Attention Without Grounding: Causal Evaluation of Visual Explanations in Medical VLMscited by 1
  34. 2025Multimodal 3D Genome Pre-trainingcited by 0
  35. 2025SD-MAD: Sign-Driven Few-shot Multi-Anomaly Detection in Medical Imagescited by 0
  36. 2025Hallucinations in medical devicescited by 0
  37. 2025PediatricsMQA: a Multi-modal Pediatrics Question Answering Benchmarkcited by 0
  38. 2025MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generationcited by 0

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025How Far Have Medical Vision-Language Models Come? A Comprehensive Benchmarking Study
  2. 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning
  3. 2026Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Models
  4. 2024Ensuring Safety and Trust: Analyzing the Risks of Large Language Models in Medicine
  5. 2025Vision Language Models in Medicine
  6. 2026MEDLAYXPLAIN: Benchmarking the Expert-Lay Gap in Medical Vision-Language Models
  7. 2025MedFoundationHub: A Lightweight and Secure Toolkit for Deploying Medical Vision Language Foundation Models
  8. 2025EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models