# Revisiting Active Learning in the Era of Vision Foundation Models

arXiv: https://arxiv.org/abs/2401.14555 · HTML: https://mlsanity.com/paper/2401.14555/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 30 of its 56 references that are in the map.
- First posted on arXiv 25 Jan 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 30 of 30)

- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark](https://mlsanity.com/paper/1910.04867/) (2019)
- [Rotation Equivariant CNNs for Digital Pathology](https://mlsanity.com/paper/1806.03962/) (2018)
- [SCAN: Learning to Classify Images without Labels](https://mlsanity.com/paper/2005.12320/) (2020)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets](https://mlsanity.com/paper/2202.02794/) (2022)
- [Contrastive Masked Autoencoders are Stronger Vision Learners](https://mlsanity.com/paper/2207.13532/) (2022)
- [Cold-start Active Learning through Self-supervised Language Modeling](https://mlsanity.com/paper/2010.09535/) (2020)
- [Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Cost](https://mlsanity.com/paper/1910.07153/) (2019)
- [Plex: Towards Reliability using Pretrained Large Model Extensions](https://mlsanity.com/paper/2207.07411/) (2022)
- [Active Learning Through a Covering Lens](https://mlsanity.com/paper/2205.11320/) (2022)
- [Active Learning by Feature Mixing](https://mlsanity.com/paper/2203.07034/) (2022)
- [Towards Robust and Reproducible Active Learning Using Neural Networks](https://mlsanity.com/paper/2002.09564/) (2020)
- [Rethinking deep active learning: Using unlabeled data at model training](https://mlsanity.com/paper/1911.08177/) (2019)
- [Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approach](https://mlsanity.com/paper/2106.02968/) (2021)
- [Active Learning Helps Pretrained Models Learn the Intended Task](https://mlsanity.com/paper/2204.08491/) (2022)
- [Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessment](https://mlsanity.com/paper/2301.10625/) (2023)
- [An empirical analysis of dropout in piecewise linear networks](https://mlsanity.com/paper/1312.6197/) (2013)
- [On the Marginal Benefit of Active Learning: Does Self-Supervision Eat Its Cake?](https://mlsanity.com/paper/2011.08121/) (2020)
- [On Initial Pools for Deep Active Learning](https://mlsanity.com/paper/2011.14696/) (2020)
- [A Simple Baseline for Low-Budget Active Learning](https://mlsanity.com/paper/2110.12033/) (2021)
- [Active Learning in Bayesian Neural Networks with Balanced Entropy Learning Principle](https://mlsanity.com/paper/2105.14559/) (2021)
- [Self-supervised Semi-supervised Learning for Data Labeling and Quality Evaluation](https://mlsanity.com/paper/2111.10932/) (2021)

## Built on by (most cited first, 14 of 14)

- [Active Prompt Learning with Vision-Language Model Priors](https://mlsanity.com/paper/2411.16722/) (2024)
- [Cleaning the Pool: Progressive Filtering of Unlabeled Pools in Deep Active Learning](https://mlsanity.com/paper/2511.22344/) (2025)
- [BoSS: A Best-of-Strategies Selector as an Oracle for Deep Active Learning](https://mlsanity.com/paper/2603.13109/) (2026)
- [Parameter-Efficient Active Learning for Foundational models](https://mlsanity.com/paper/2406.09296/) (2024)
- [No Free Lunch in Active Learning: LLM Embedding Quality Dictates Query Strategy Success](https://mlsanity.com/paper/2506.01992/) (2025)
- [Finally Outshining the Random Baseline: A Simple and Effective Solution for Active Learning in 3D Biomedical Imaging](https://mlsanity.com/paper/2601.13677/) (2026)
- [Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibration](https://mlsanity.com/paper/2507.21521/) (2025)
- [Combining Discrepancy-Confusion Uncertainty and Calibration Diversity for Active Fine-Grained Image Classification](https://mlsanity.com/paper/2509.24181/) (2025)
- [Bi-CoG: Bi-Consistency-Guided Self-Training for Vision-Language Models](https://mlsanity.com/paper/2510.20477/) (2025)
- [Active Learning with Task-Driven Representations for Messy Pools](https://mlsanity.com/paper/2510.25926/) (2025)
- [Conformal Cross-Modal Active Learning](https://mlsanity.com/paper/2603.23159/) (2026)
- [Active Learning with Foundation Model Priors: Efficient Learning under Class Imbalance](https://mlsanity.com/paper/2606.07630/) (2026)
- [Unsupervised Transfer Clustering for Mitigating Cold Start in Active Prompt Learning](https://mlsanity.com/paper/2609.05636/) (2026)
- [Measure Less, Know More: Self-Supervised Test-Time Feature Acquisition](https://mlsanity.com/paper/2610.03454/) (2026)

## Abstract (arXiv)

Foundation vision or vision-language models are trained on large unlabeled or noisy data and learn robust representations that can achieve impressive zero- or few-shot performance on diverse tasks. Given these properties, they are a natural fit for active learning (AL), which aims to maximize labeling efficiency. However, the full potential of foundation models has not been explored in the context of AL, specifically in the low-budget regime. In this work, we evaluate how foundation models influence three critical components of effective AL, namely, 1) initial labeled pool selection, 2) ensuring diverse sampling, and 3) the trade-off between representative and uncertainty sampling. We systematically study how the robust representations of foundation models (DINOv2, OpenCLIP) challenge existing findings in active learning. Our observations inform the principled construction of a new simple and elegant AL strategy that balances uncertainty estimated via dropout with sample diversity. We extensively test our strategy on many challenging image classification benchmarks, including natural images as well as out-of-domain biomedical images that are relatively understudied in the AL literature. We also provide a highly performant and efficient implementation of modern AL strategies (including our method) at https://github.com/sanketx/AL-foundation-models.