mlsanityData to 7 Oct 2026

A Survey on Trustworthiness in Foundation Models for Medical Image Analysis

arXiv 2407.15851 ↗First posted 3 Jul 2024cs.CV cs.AI cs.CYcited by 16

Not on the map. Why?

Abstract (arXiv)

The rapid advancement of foundation models in medical imaging represents a significant leap toward enhancing diagnostic accuracy and personalized treatment. However, the deployment of foundation models in healthcare necessitates a rigorous examination of their trustworthiness, encompassing privacy, robustness, reliability, explainability, and fairness. The current body of survey literature on foundation models in medical imaging reveals considerable gaps, particularly in the area of trustworthiness. Additionally, existing surveys on the trustworthiness of foundation models do not adequately address their specific variations and applications within the medical imaging domain. This survey aims to fill that gap by presenting a novel taxonomy of foundation models used in medical imaging and analyzing the key motivations for ensuring their trustworthiness. We review current research on foundation models in major medical imaging applications, focusing on segmentation, medical report generation, medical question and answering (Q&A), and disease diagnosis. These areas are highlighted because they have seen a relatively mature and substantial number of foundation models compared to other applications. We focus on literature that discusses trustworthiness in medical image analysis manuscripts. We explore the complex challenges of building trustworthy foundation models for each application, summarizing current concerns and strategies for enhancing trustworthiness. Furthermore, we examine the potential of these models to revolutionize patient care. Our analysis underscores the imperative for advancing towards trustworthy AI in medical image analysis, advocating for a balanced approach that fosters innovation while ensuring ethical and equitable healthcare delivery.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 78 of its 97 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

78 of 97 references.

References · 78

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  8. 2023Segment Anythingcited by 8,158
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  11. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
38 more
  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2022Visual Prompt Tuningcited by 1,674
  3. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  4. 2018Object Hallucination in Image Captioningcited by 661
  5. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  6. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 564
  7. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  8. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  9. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  10. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  11. 2022Visual Classification via Description from Large Language Modelscited by 331
  12. 2023Explainability for Large Language Models: A Surveycited by 331
  13. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  14. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  15. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  16. 20192017 Robotic Instrument Segmentation Challengecited by 166
  17. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  18. 2023Interactive and Explainable Region-guided Radiology Report Generationcited by 139
  19. 2023ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Modelscited by 116
  20. 2024BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at oncecited by 107
  21. 2023NEFTune: Noisy Embeddings Improve Instruction Finetuningcited by 99
  22. 2021Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain?cited by 85
  23. 2021Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methodscited by 83
  24. 2024A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretationcited by 79
  25. 2023Can GPT-4V(ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosiscited by 77
  26. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  27. 2023Foundational Models in Medical Imaging: A Comprehensive Survey and Future Visioncited by 74
  28. 2023MAIRA-1: A specialised large multimodal model for radiology report generationcited by 72
  29. 2024FairCLIP: Harnessing Fairness in Vision-Language Learningcited by 69
  30. 2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Modelscited by 64
  31. 2023SAM.MD: Zero-shot medical image segmentation capabilities of the Segment Anything Modelcited by 61
  32. 2023XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Modelscited by 61
  33. 2023Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Modelscited by 58
  34. 2023LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generationcited by 57
  35. 2024Detecting and Evaluating Medical Hallucinations in Large Vision Language Modelscited by 57
  36. 2022Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priorscited by 48
  37. 2024From Understanding to Utilization: A Survey on Explainability for Large Language Modelscited by 47
  38. 2024FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Modelscited by 47

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 11; 2026: 3.

16 papers.

Sort:

By area:

  1. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 84
  2. 2025Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Reviewcited by 8
  3. 2025MOTOR: Multimodal Optimal Transport via Grounded Retrieval in Medical Visual Question Answeringcited by 5
  4. 2024Efficient MedSAMs: Segment Anything in Medical Images on Laptopcited by 4
  5. 2025Where are we with calibration under dataset shift in image classification?cited by 4
  6. 2025Fair Foundation Models for Medical Image Analysis: Challenges and Perspectivescited by 3
  7. 2026Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Benchcited by 3
  8. 2025MedIQA: A Scalable Foundation Model for Prompt-Driven Medical Image Quality Assessmentcited by 2
  9. 2025Federated Learning for Large Models in Medical Imaging: A Comprehensive Reviewcited by 2
  10. 2025Foundation Models in Biomedical Imaging: Turning Hype into Realitycited by 2
  11. 2025Brain Imaging Foundation Models, Are We There Yet? A Systematic Review of Foundation Models for Brain Imaging and Biomedical Researchcited by 0
  12. 2025UNICON: UNIfied CONtinual Learning for Medical Foundational Modelscited by 0
4 more
  1. 2025MAFM^3: Modular Adaptation of Foundation Models for Multi-Modal Medical AIcited by 0
  2. 2025XAI-Driven Diagnosis of Generalization Failure in State-Space Cerebrovascular Segmentation Models: A Case Study on Domain Shift Between RSNA and TopCoW Datasetscited by 0
  3. 2026Analogical Reasoning as a Doctor: A Foundation Model for Gastrointestinal Endoscopy Diagnosiscited by 0
  4. 2026Hallucinations and Constraints : Regulating surgical workflow recognition beyond accuracycited by 0