# Less Could Be Better: Parameter-efficient Fine-tuning Advances Medical Vision Foundation Models

arXiv: https://arxiv.org/abs/2401.12215 · HTML: https://mlsanity.com/paper/2401.12215/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 22 Jan 2024.

## Builds on (most cited first, 17 of 17)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://mlsanity.com/paper/2205.05638/) (2022)
- [VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks](https://mlsanity.com/paper/2112.06825/) (2021)
- [AIM: Adapting Image Models for Efficient Video Action Recognition](https://mlsanity.com/paper/2302.03024/) (2023)
- [Advancing Radiograph Representation Learning with Masked Record Modeling](https://mlsanity.com/paper/2301.13155/) (2023)
- [Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity](https://mlsanity.com/paper/2305.08252/) (2023)
- [A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics](https://mlsanity.com/paper/2306.00864/) (2023)
- [MeLo: Low-rank Adaptation is Better than Fine-tuning for Medical Image Diagnosis](https://mlsanity.com/paper/2311.08236/) (2023)
- [Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image Analysis](https://mlsanity.com/paper/2302.05615/) (2023)
- [Difficulty-aware Glaucoma Classification with Multi-Rater Consensus Modeling](https://mlsanity.com/paper/2007.14848/) (2020)

## Built on by (most cited first, 18 of 18)

- [MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning](https://mlsanity.com/paper/2502.19634/) (2025)
- [BAPLe: Backdoor Attacks on Medical Foundational Models using Prompt Learning](https://mlsanity.com/paper/2408.07440/) (2024)
- [Kvasir-VQA-x1: A Multimodal Dataset for Medical Reasoning and Robust MedVQA in Gastrointestinal Endoscopy](https://mlsanity.com/paper/2506.09958/) (2025)
- [Foundation Models in Medical Imaging: A Review and Outlook](https://mlsanity.com/paper/2506.09095/) (2025)
- [Efficient Medical Vision-Language Alignment Through Adapting Masked Vision Models](https://mlsanity.com/paper/2506.08990/) (2025)
- [Medical Knowledge Intervention Prompt Tuning for Medical Image Classification](https://mlsanity.com/paper/2511.12639/) (2025)
- [Benchmarking Foundation Models and Parameter-Efficient Fine-Tuning for Prognosis Prediction in Medical Imaging](https://mlsanity.com/paper/2506.18434/) (2025)
- [Low-Rank Adaption on Transformer-based Oriented Object Detector for Satellite Onboard Processing of Remote Sensing Images](https://mlsanity.com/paper/2406.02385/) (2024)
- [Multi LoRA Meets Vision: Merging multiple adapters to create a multi task model](https://mlsanity.com/paper/2411.14064/) (2024)
- [RAU: Reference-based Anatomical Understanding with Vision Language Models](https://mlsanity.com/paper/2509.22404/) (2025)
- [Robustness Tokens: Towards Adversarial Robustness of Transformers](https://mlsanity.com/paper/2503.10191/) (2025)
- [TAP: Two-Stage Adaptive Personalization of Multi-Task and Multi-Modal Foundation Models in Federated Learning](https://mlsanity.com/paper/2509.26524/) (2025)
- [Fine-tuning a multimodal large language model for clinician-grade autism behavioral scoring from short home videos](https://mlsanity.com/paper/2606.27484/) (2026)
- [Efficient Self-Supervised Adaptation for Medical Image Analysis](https://mlsanity.com/paper/2503.18873/) (2025)
- [Adapting Foundation Vision-Language Models to Medical Diagnosis via Query-Driven Expert Bridging](https://mlsanity.com/paper/2505.21698/) (2025)
- [TAP-SLF: Parameter-Efficient Adaptation of Vision Foundation Models for Multi-Task Ultrasound Image Analysis](https://mlsanity.com/paper/2603.00433/) (2026)
- [CMRVision: A Foundation Model for Cardiac MR Image Analysis](https://mlsanity.com/paper/2609.01308/) (2026)
- [Beyond Training from Scratch: Foundation Models for Data-Efficient and Generalizable Cardiac MRI Reconstruction](https://mlsanity.com/paper/2610.08109/) (2026)

## Abstract (arXiv)

Parameter-efficient fine-tuning (PEFT) that was initially developed for exploiting pre-trained large language models has recently emerged as an effective approach to perform transfer learning on computer vision tasks. However, the effectiveness of PEFT on medical vision foundation models is still unclear and remains to be explored. As a proof of concept, we conducted a detailed empirical study on applying PEFT to chest radiography foundation models. Specifically, we delved into LoRA, a representative PEFT method, and compared it against full-parameter fine-tuning (FFT) on two self-supervised radiography foundation models across three well-established chest radiograph datasets. Our results showed that LoRA outperformed FFT in 13 out of 18 transfer learning tasks by at most 2.9% using fewer than 1% tunable parameters. Combining LoRA with foundation models, we set up new state-of-the-art on a range of data-efficient learning tasks, such as an AUROC score of 80.6% using 1% labeled data on NIH ChestX-ray14. We hope this study can evoke more attention from the community in the use of PEFT for transfer learning on medical imaging tasks. Code and models are available at https://github.com/RL4M/MED-PEFT.