# Adapting Medical Vision Foundation Models for Volumetric Medical Image Segmentation via Active Learning and Selective Semi-supervised Fine-tuning

arXiv: https://arxiv.org/abs/2509.10784 · HTML: https://mlsanity.com/paper/2509.10784/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 13 Sep 2025.

## Builds on (most cited first, 23 of 23)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation](https://mlsanity.com/paper/1706.04737/) (2017)
- [On the Challenges and Perspectives of Foundation Models for Medical Image Analysis](https://mlsanity.com/paper/2306.05705/) (2023)
- [ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentation](https://mlsanity.com/paper/1911.11789/) (2019)
- [Vision Foundation Models for Computed Tomography](https://mlsanity.com/paper/2501.09001/) (2025)
- [Diminishing Uncertainty within the Training Pool: Active Learning for Medical Image Segmentation](https://mlsanity.com/paper/2101.02323/) (2021)
- [Multi-Anchor Active Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2108.08012/) (2021)
- [DSAL: Deeply Supervised Active Learning from Strong and Weak Labelers for Biomedical Image Segmentation](https://mlsanity.com/paper/2101.09057/) (2021)
- [Unsupervised Selective Labeling for More Effective Semi-Supervised Learning](https://mlsanity.com/paper/2110.03006/) (2021)
- [Dual-Reference Source-Free Active Domain Adaptation for Nasopharyngeal Carcinoma Tumor Segmentation across Multiple Hospitals](https://mlsanity.com/paper/2309.13401/) (2023)
- [Triad: Vision Foundation Model for 3D Magnetic Resonance Imaging](https://mlsanity.com/paper/2502.14064/) (2025)
- [Automated Muscle Segmentation from Clinical CT using Bayesian U-Net for Personalized Musculoskeletal Modeling](https://mlsanity.com/paper/1907.08915/) (2019)
- [Active learning for medical image segmentation with stochastic batches](https://mlsanity.com/paper/2301.07670/) (2023)
- [Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer](https://mlsanity.com/paper/2311.12905/) (2023)
- [3D Foundation Model for Generalizable Disease Detection in Head Computed Tomography](https://mlsanity.com/paper/2502.02779/) (2025)
- [Advancing UWF-SLO Vessel Segmentation with Source-Free Active Domain Adaptation and a Novel Multi-Center Dataset](https://mlsanity.com/paper/2406.13645/) (2024)
- [SALAD: Source-free Active Label-Agnostic Domain Adaptation for Classification, Segmentation and Detection](https://mlsanity.com/paper/2205.12840/) (2022)
- [Brain Foundation Models with Hypergraph Dynamic Adapter for Brain Disease Analysis](https://mlsanity.com/paper/2505.00627/) (2025)
- [An Uncertainty-guided Tiered Self-training Framework for Active Source-free Domain Adaptation in Prostate Segmentation](https://mlsanity.com/paper/2407.02893/) (2024)
- [MRAnnotator: multi-Anatomy and many-Sequence MRI segmentation of 44 structures](https://mlsanity.com/paper/2402.01031/) (2024)
- [Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2305.00773/) (2023)

## Abstract (arXiv)

Medical vision foundation models remain limited in downstream tasks, particularly volumetric medical image segmentation. While fine-tuning on labeled target-domain data improves performance, existing approaches typically rely on randomly selected samples, which may fail to identify the most informative data and thus hinder adaptation. To address the limitations, we propose an Active Selective Semi-supervised Fine-tuning framework for efficient adaptation of Med-VFMs to generalize across volumetric medical image segmentation. ASSFT integrates a novel active learning strategy with selective semi-supervised learning to maximize adaptation performance under a limited annotation budget, without requiring access to source data. Specifically, we introduce an Active Test-Time Sample Query strategy that identifies informative samples from the target domain using two complementary query metrics: Diversified Knowledge Divergence and Anatomical Segmentation Difficulty. DKD quantifies both the knowledge gap between pre-training and target domains and the semantic diversity within the target dataset, enabling the selection of samples that contain previously unlearned knowledge while maintaining intra-domain diversity. ASD estimates the segmentation difficulty of target anatomical structures by measuring predictive uncertainty within foreground regions of interest, allowing the model to prioritize samples with complex anatomical patterns rather than those dominated by background uncertainty. Second, we propose a Selective Semi-supervised Fine-tuning strategy to further improve adaptation performance by leveraging unlabeled target samples. Instead of utilizing all pseudo-labeled data, the proposed method selectively incorporates reliable unlabeled samples based on predictive confidence and semantic distance to labeled samples, enabling stable semi-supervised training while avoiding noisy pseudo-labels.