# Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity

arXiv: https://arxiv.org/abs/2305.08252 · HTML: https://mlsanity.com/paper/2305.08252/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 97 references that are in the map.
- First posted on arXiv 14 May 2023.

## Builds on (most cited first, 30 of 58)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Towards a Unified View of Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2110.04366/) (2021)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning](https://mlsanity.com/paper/2012.13255/) (2020)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)

## Built on by (most cited first, 30 of 56)

- [Improving Tuning-Free Real Image Editing with Proximal Guidance](https://mlsanity.com/paper/2306.05414/) (2023)
- [Evaluating General Purpose Vision Foundation Models for Medical Image Analysis: An Experimental Study of DINOv2 on Radiology Benchmarks](https://mlsanity.com/paper/2312.02366/) (2023)
- [Artificial General Intelligence for Medical Imaging Analysis](https://mlsanity.com/paper/2306.05480/) (2023)
- [How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model](https://mlsanity.com/paper/2404.09957/) (2024)
- [Data-Centric Foundation Models in Computational Healthcare: A Survey](https://mlsanity.com/paper/2401.02458/) (2024)
- [Parameter Efficient Fine Tuning: A Comprehensive Analysis Across Applications](https://mlsanity.com/paper/2404.13506/) (2024)
- [SAM-PARSER: Fine-tuning SAM Efficiently by Parameter Space Reconstruction](https://mlsanity.com/paper/2308.14604/) (2023)
- [FairTune: Optimizing Parameter Efficient Fine Tuning for Fairness in Medical Image Analysis](https://mlsanity.com/paper/2310.05055/) (2023)
- [Less Could Be Better: Parameter-efficient Fine-tuning Advances Medical Vision Foundation Models](https://mlsanity.com/paper/2401.12215/) (2024)
- [Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality Forecasting](https://mlsanity.com/paper/2405.10216/) (2024)
- [PRISM: High-Resolution & Precise Counterfactual Medical Image Generation using Language-guided Stable Diffusion](https://mlsanity.com/paper/2503.00196/) (2025)
- [ConvLoRA and AdaBN based Domain Adaptation via Self-Training](https://mlsanity.com/paper/2402.04964/) (2024)
- [Beyond LoRA: Exploring Efficient Fine-Tuning Techniques for Time Series Foundational Models](https://mlsanity.com/paper/2409.11302/) (2024)
- [Foundation Models in Medical Imaging: A Review and Outlook](https://mlsanity.com/paper/2506.09095/) (2025)
- [Fine-grained Prompt Tuning: A Parameter and Memory Efficient Transfer Learning Method for High-resolution Medical Image Classification](https://mlsanity.com/paper/2403.07576/) (2024)
- [Embedded Visual Prompt Tuning](https://mlsanity.com/paper/2407.01003/) (2024)
- [LoRTA: Low Rank Tensor Adaptation of Large Language Models](https://mlsanity.com/paper/2410.04060/) (2024)
- [BMFT: Achieving Fairness via Bias-based Weight Masking Fine-tuning](https://mlsanity.com/paper/2408.06890/) (2024)
- [CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of Synthetic Chest Radiographs](https://mlsanity.com/paper/2505.10496/) (2025)
- [Efficient Medical Vision-Language Alignment Through Adapting Masked Vision Models](https://mlsanity.com/paper/2506.08990/) (2025)
- [MemControl: Mitigating Memorization in Diffusion Models via Automated Parameter Selection](https://mlsanity.com/paper/2405.19458/) (2024)
- [Benchmarking Foundation Models and Parameter-Efficient Fine-Tuning for Prognosis Prediction in Medical Imaging](https://mlsanity.com/paper/2506.18434/) (2025)
- [Sparsity- and Hybridity-Inspired Visual Parameter-Efficient Fine-Tuning for Medical Diagnosis](https://mlsanity.com/paper/2405.17877/) (2024)
- [Curriculum Fine-tuning of Vision Foundation Model for Medical Image Classification Under Label Noise](https://mlsanity.com/paper/2412.00150/) (2024)
- [The Devil is in the Prompts: De-Identification Traces Enhance Memorization Risks in Synthetic Chest X-Ray Generation](https://mlsanity.com/paper/2502.07516/) (2025)
- [Exploiting Mixture-of-Experts Redundancy Unlocks Multimodal Generative Abilities](https://mlsanity.com/paper/2503.22517/) (2025)
- [RAU: Reference-based Anatomical Understanding with Vision Language Models](https://mlsanity.com/paper/2509.22404/) (2025)
- [Adaptation of Foundation Models for Medical Image Analysis: Strategies, Challenges, and Future Directions](https://mlsanity.com/paper/2511.01284/) (2025)
- [SkinCLIP-VL: Consistency-Aware Vision-Language Learning for Multimodal Skin Cancer Diagnosis](https://mlsanity.com/paper/2603.21010/) (2026)
- [Boosting Memory Efficiency in Transfer Learning for High-Resolution Medical Image Classification](https://mlsanity.com/paper/2408.02426/) (2024)

## Abstract (arXiv)

Foundation models have significantly advanced medical image analysis through the pre-train fine-tune paradigm. Among various fine-tuning algorithms, Parameter-Efficient Fine-Tuning (PEFT) is increasingly utilized for knowledge transfer across diverse tasks, including vision-language and text-to-image generation. However, its application in medical image analysis is relatively unexplored due to the lack of a structured benchmark for evaluating PEFT methods. This study fills this gap by evaluating 17 distinct PEFT algorithms across convolutional and transformer-based networks on image classification and text-to-image generation tasks using six medical datasets of varying size, modality, and complexity. Through a battery of over 700 controlled experiments, our findings demonstrate PEFT's effectiveness, particularly in low data regimes common in medical imaging, with performance gains of up to 22% in discriminative and generative tasks. These recommendations can assist the community in incorporating PEFT into their workflows and facilitate fair comparisons of future PEFT methods, ensuring alignment with advancements in other areas of machine learning and AI.