# MedCAL-Bench: A Comprehensive Benchmark on Cold-Start Active Learning with Foundation Models for Medical Image Analysis

arXiv: https://arxiv.org/abs/2508.03441 · HTML: https://mlsanity.com/paper/2508.03441/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation](https://mlsanity.com/paper/1706.04737/) (2017)
- [Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets](https://mlsanity.com/paper/2202.02794/) (2022)
- [Cold-start Active Learning through Self-supervised Language Modeling](https://mlsanity.com/paper/2010.09535/) (2020)
- [Active Learning Through a Covering Lens](https://mlsanity.com/paper/2205.11320/) (2022)
- [A Framework and Benchmark for Deep Batch Active Learning for Regression](https://mlsanity.com/paper/2203.09410/) (2022)
- [Diminishing Uncertainty within the Training Pool: Active Learning for Medical Image Segmentation](https://mlsanity.com/paper/2101.02323/) (2021)
- [Decomposing Disease Descriptions for Enhanced Pathology Detection: A Multi-Aspect Vision-Language Pre-training Framework](https://mlsanity.com/paper/2403.07636/) (2024)
- [Making Your First Choice: To Address Cold Start Problem in Vision Active Learning](https://mlsanity.com/paper/2210.02442/) (2022)
- [A Cross-Domain Benchmark for Active Learning](https://mlsanity.com/paper/2408.00426/) (2024)
- [BaSAL: Size-Balanced Warm Start Active Learning for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2310.08035/) (2023)
- [Data efficient deep learning for medical image analysis: A survey](https://mlsanity.com/paper/2310.06557/) (2023)
- [Bridging Diversity and Uncertainty in Active learning with Self-Supervised Pre-Training](https://mlsanity.com/paper/2403.03728/) (2024)
- [An Uncertainty-guided Tiered Self-training Framework for Active Source-free Domain Adaptation in Prostate Segmentation](https://mlsanity.com/paper/2407.02893/) (2024)

## Built on by (most cited first, 1 of 1)

- [Dataset-Aware Cold-Start Active Learning for Annotation-Efficient 3D Medical Image Segmentation](https://mlsanity.com/paper/2606.20765/) (2026)

## Abstract (arXiv)

Cold-Start Active Learning (CSAL) aims to select informative samples for annotation without prior knowledge, which is important for improving annotation efficiency and model performance under a limited annotation budget in medical image analysis. Most existing CSAL methods rely on Self-Supervised Learning (SSL) on the target dataset for feature extraction, which is inefficient and limited by insufficient feature representation. Recently, pre-trained Foundation Models (FMs) have shown powerful feature extraction ability with a potential for better CSAL. However, this paradigm has been rarely investigated, with a lack of benchmarks for comparison of FMs in CSAL tasks. To this end, we propose MedCAL-Bench, the first systematic FM-based CSAL benchmark for medical image analysis. We evaluate 14 FMs and 7 CSAL strategies across 7 datasets under different annotation budgets, covering classification and segmentation tasks from diverse medical modalities. It is also the first CSAL benchmark that evaluates both the feature extraction and sample selection stages. Our experimental results reveal that: 1) Most FMs are effective feature extractors for CSAL, with DINO family performing the best in segmentation; 2) The performance differences of these FMs are large in segmentation tasks, while small for classification; 3) Different sample selection strategies should be considered in CSAL on different datasets, with Active Learning by Processing Surprisal (ALPS) performing the best in segmentation while RepDiv leading for classification. The code is available at https://github.com/HiLab-git/MedCAL-Bench.