# Active Learning Through a Covering Lens

arXiv: https://arxiv.org/abs/2205.11320 · HTML: https://mlsanity.com/paper/2205.11320/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 61 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [SCAN: Learning to Classify Images without Labels](https://mlsanity.com/paper/2005.12320/) (2020)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets](https://mlsanity.com/paper/2202.02794/) (2022)
- [Cold-start Active Learning through Self-supervised Language Modeling](https://mlsanity.com/paper/2010.09535/) (2020)
- [Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Cost](https://mlsanity.com/paper/1910.07153/) (2019)
- [Discriminative Active Learning](https://mlsanity.com/paper/1907.06347/) (2019)
- [Deep Active Learning: Unified and Principled Method for Query and Training](https://mlsanity.com/paper/1911.09162/) (2019)
- [Diverse mini-batch Active Learning](https://mlsanity.com/paper/1901.05954/) (2019)
- [Deep Active Learning over the Long Tail](https://mlsanity.com/paper/1711.00941/) (2017)
- [Towards Robust and Reproducible Active Learning Using Neural Networks](https://mlsanity.com/paper/2002.09564/) (2020)
- [Reducing Label Effort: Self-Supervised meets Active Learning](https://mlsanity.com/paper/2108.11458/) (2021)
- [Rethinking deep active learning: Using unlabeled data at model training](https://mlsanity.com/paper/1911.08177/) (2019)
- [Parting with Illusions about Deep Active Learning](https://mlsanity.com/paper/1912.05361/) (2019)
- [Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approach](https://mlsanity.com/paper/2106.02968/) (2021)
- [Addressing the Item Cold-start Problem by Attribute-driven Active Learning](https://mlsanity.com/paper/1805.09023/) (2018)
- [On the Marginal Benefit of Active Learning: Does Self-Supervision Eat Its Cake?](https://mlsanity.com/paper/2011.08121/) (2020)
- [On Initial Pools for Deep Active Learning](https://mlsanity.com/paper/2011.14696/) (2020)

## Built on by (most cited first, 30 of 66)

- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [A comprehensive survey on deep active learning in medical image analysis](https://mlsanity.com/paper/2310.14230/) (2023)
- [Making Your First Choice: To Address Cold Start Problem in Vision Active Learning](https://mlsanity.com/paper/2210.02442/) (2022)
- [LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning](https://mlsanity.com/paper/2306.09910/) (2023)
- [Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks](https://mlsanity.com/paper/2311.10609/) (2023)
- [How to Select Which Active Learning Strategy is Best Suited for Your Specific Problem and Budget](https://mlsanity.com/paper/2306.03543/) (2023)
- [Generalized Coverage for More Robust Low-Budget Active Learning](https://mlsanity.com/paper/2407.12212/) (2024)
- [Uncertainty Herding: One Active Learning Method for All Label Budgets](https://mlsanity.com/paper/2412.20644/) (2024)
- [FreeAL: Towards Human-Free Active Learning in the Era of Large Language Models](https://mlsanity.com/paper/2311.15614/) (2023)
- [Revisiting Active Learning in the Era of Vision Foundation Models](https://mlsanity.com/paper/2401.14555/) (2024)
- [Making Better Use of Unlabelled Data in Bayesian Active Learning](https://mlsanity.com/paper/2404.17249/) (2024)
- [BaSAL: Size-Balanced Warm Start Active Learning for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2310.08035/) (2023)
- [Querying Easily Flip-flopped Samples for Deep Active Learning](https://mlsanity.com/paper/2401.09787/) (2024)
- [A Unified Approach Towards Active Learning and Out-of-Distribution Detection](https://mlsanity.com/paper/2405.11337/) (2024)
- [Predicting the Susceptibility of Examples to Catastrophic Forgetting](https://mlsanity.com/paper/2406.09935/) (2024)
- [Joint Out-of-Distribution Filtering and Data Discovery Active Learning](https://mlsanity.com/paper/2503.02491/) (2025)
- [You Never Get a Second Chance To Make a Good First Impression: Seeding Active Learning for 3D Semantic Segmentation](https://mlsanity.com/paper/2304.11762/) (2023)
- [Bridging Diversity and Uncertainty in Active learning with Self-Supervised Pre-Training](https://mlsanity.com/paper/2403.03728/) (2024)
- [Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classification](https://mlsanity.com/paper/2404.08981/) (2024)
- [Deep Active Learning in the Open World](https://mlsanity.com/paper/2411.06353/) (2024)
- [Active Learning via Classifier Impact and Greedy Selection for Interactive Image Retrieval](https://mlsanity.com/paper/2412.02310/) (2024)
- [Feature Alignment: Rethinking Efficient Active Learning via Proxy in the Context of Pre-trained Models](https://mlsanity.com/paper/2403.01101/) (2024)
- [NTKCPL: Active Learning on Top of Self-Supervised Model by Estimating True Coverage](https://mlsanity.com/paper/2306.04099/) (2023)
- [Learning to Rank for Active Learning via Multi-Task Bilevel Optimization](https://mlsanity.com/paper/2310.17044/) (2023)
- [Class Balance Matters to Active Class-Incremental Learning](https://mlsanity.com/paper/2412.06642/) (2024)
- [Enhancing Cost Efficiency in Active Learning with Candidate Set Query](https://mlsanity.com/paper/2502.06209/) (2025)
- [Active Learning with a Noisy Annotator](https://mlsanity.com/paper/2504.04506/) (2025)
- [Enhancing Treatment Effect Estimation via Active Learning: A Counterfactual Covering Perspective](https://mlsanity.com/paper/2505.05242/) (2025)
- [SUPClust: Active Learning at the Boundaries](https://mlsanity.com/paper/2403.03741/) (2024)
- [DCoM: Active Learning for All Learners](https://mlsanity.com/paper/2407.01804/) (2024)

## Abstract (arXiv)

Deep active learning aims to reduce the annotation cost for the training of deep models, which is notoriously data-hungry. Until recently, deep active learning methods were ineffectual in the low-budget regime, where only a small number of examples are annotated. The situation has been alleviated by recent advances in representation and self-supervised learning, which impart the geometry of the data representation with rich information about the points. Taking advantage of this progress, we study the problem of subset selection for annotation through a "covering" lens, proposing ProbCover - a new active learning algorithm for the low budget regime, which seeks to maximize Probability Coverage. We then describe a dual way to view the proposed formulation, from which one can derive strategies suitable for the high budget regime of active learning, related to existing methods like Coreset. We conclude with extensive experiments, evaluating ProbCover in the low-budget regime. We show that our principled active learning strategy improves the state-of-the-art in the low-budget regime in several image recognition benchmarks. This method is especially beneficial in the semi-supervised setting, allowing state-of-the-art semi-supervised methods to match the performance of fully supervised methods, while using much fewer labels nonetheless. Code is available at https://github.com/avihu111/TypiClust.