Adaptation of Foundation Models for Medical Image Analysis: Strategies, Challenges, and Future Directions
Not on the map. Why?
Abstract (arXiv)
Foundation models (FMs) have emerged as a transformative paradigm in medical image analysis, offering the potential to provide generalizable, task-agnostic solutions across a wide range of clinical tasks and imaging modalities. Their capacity to learn transferable representations from large-scale data has the potential to address the limitations of conventional task-specific models. However, adaptation of FMs to real-world clinical practice remains constrained by key challenges, including domain shifts, limited availability of high-quality annotated data, substantial computational demands, and strict privacy requirements. This review presents a comprehensive assessment of strategies for adapting FMs to the specific demands of medical imaging. We examine approaches such as supervised fine-tuning, domain-specific pretraining, parameter-efficient fine-tuning, self-supervised learning, hybrid methods, and multimodal or cross-modal frameworks. For each, we evaluate reported performance gains, clinical applicability, and limitations, while identifying trade-offs and unresolved challenges that prior reviews have often overlooked. Beyond these established techniques, we also highlight emerging directions aimed at addressing current gaps. These include continual learning to enable dynamic deployment, federated and privacy-preserving approaches to safeguard sensitive data, hybrid self-supervised learning to enhance data efficiency, data-centric pipelines that combine synthetic generation with human-in-the-loop validation, and systematic benchmarking to assess robust generalization under real-world clinical variability. By outlining these strategies and associated research gaps, this review provides a roadmap for developing adaptive, trustworthy, and clinically integrated FMs capable of meeting the demands of real-world medical imaging.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 100 of its 143 references that are in the map. First posted on arXiv 3 Nov 2025.
Builds on
100 of 143 references.
References · 100
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2023GPT-4 Technical Reportcited by 18,396
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2023Segment Anythingcited by 8,234
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
38 more
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
- 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
- 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
- 2014How transferable are features in deep neural networks?cited by 1,225
- 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,222
- 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,186
- 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
- 2018Experience Replay for Continual Learningcited by 1,095
- 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
- 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
- 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
- 2020Self-supervised Learning: Generative or Contrastivecited by 694
- 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 690
- 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
- 2023A Comprehensive Overview of Large Language Modelscited by 648
- 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 641
- 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 570
- 2020A Survey on Contrastive Self-supervised Learningcited by 554
- 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
- 2023Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentationcited by 490
- 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 460
- 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 422
- 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 368
- 2023AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuningcited by 357
- 2021Big Self-Supervised Models Advance Medical Image Classificationcited by 320
- 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
- 2023Customized Segment Anything Model for Medical Image Segmentationcited by 280
- 2017Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?cited by 221
- 2017On the Automatic Generation of Medical Imaging Reportscited by 216
- 2021Training Neural Networks with Fixed Sparse Maskscited by 214
- 2022Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Modelscited by 180
- 2023Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuningcited by 173
- 2021UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuningcited by 162
50 most cited of 100 All 100 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Geometry-Consistent Endoscopic Representations for Image-Guided Navigation via Structured Foundation Model Adaptationcited by 0
- 2026CoM3eT: A foundation model for medical image analysis through federated, multidimensional context integrationcited by 0
- 2026FAN-LoRA: A Fourier-Adaptive Nonlinear Low-Rank Adaptor for Medical Foundation Model Domain Adaptationcited by 0
- 2026FAN-LoRA: A Fourier-Adaptive Nonlinear Low-Rank Adaptor for Medical Foundation Model Domain Adaptation27 Aug 2026
- 2026CoM3eT: A foundation model for medical image analysis through federated, multidimensional context integration17 Aug 2026
- 2026Geometry-Consistent Endoscopic Representations for Image-Guided Navigation via Structured Foundation Model Adaptation15 Jun 2026