mlsanityData to 7 Oct 2026

A Simple Baseline for Low-Budget Active Learning

arXiv 2110.12033 ↗First posted 22 Oct 2021cs.CV cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Active learning focuses on choosing a subset of unlabeled data to be labeled. However, most such methods assume that a large subset of the data can be annotated. We are interested in low-budget active learning where only a small subset (e.g., 0.2% of ImageNet) can be annotated. Instead of proposing a new query strategy to iteratively sample batches of unlabeled data given an initial pool, we learn rich features by an off-the-shelf self-supervised learning method only once, and then study the effectiveness of different sampling strategies given a low labeling budget on a variety of datasets including ImageNet. We show that although the state-of-the-art active learning methods work well given a large labeling budget, a simple K-means clustering algorithm can outperform them on low budgets. We believe this method can be used as a simple baseline for low-budget active learning on image classification. Code is available at: https://github.com/UCDvision/low-budget-al

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 41 of its 61 references that are in the map. First posted on arXiv 22 Oct 2021.

Builds on

41 of 61 references.

References · 41

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  7. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  8. 2017Billion-scale similarity search with GPUscited by 2,477
  9. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  10. 2013Describing Textures in the Wildcited by 2,110
  11. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  12. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
29 more
  1. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  2. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  3. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  4. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  5. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  6. 2019Contrastive Multiview Codingcited by 1,297
  7. 2016Colorful Image Colorizationcited by 1,113
  8. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  9. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  10. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  11. 2016Learning to learn by gradient descent by gradient descentcited by 616
  12. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  13. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  14. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  15. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  16. 2019Variational Adversarial Active Learningcited by 284
  17. 2018Adversarial Active Learning for Deep Networks: a Margin Based Approachcited by 155
  18. 2017Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentationcited by 145
  19. 2017Representation Learning by Learning to Countcited by 120
  20. 2019Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Costcited by 114
  21. 2019Deep Active Learning: Unified and Principled Method for Query and Trainingcited by 92
  22. 2020CompRess: Self-Supervised Learning by Compressing Representationscited by 61
  23. 2020State-Relabeling Adversarial Active Learningcited by 60
  24. 2020FeatMatch: Feature-Based Augmentation for Semi-Supervised Learningcited by 57
  25. 2021Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approachcited by 34
  26. 2018Cost-Sensitive Active Learning for Intracranial Hemorrhage Detectioncited by 28
  27. 2020On Initial Pools for Deep Active Learningcited by 18
  28. 2020Minimax Active Learningcited by 18
  29. 2019Adversarial Representation Active Learningcited by 7

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 6; 2024: 3; 2025: 1.

15 papers.

Sort:

By area:

  1. 2022Active Learning on a Budget: Opposite Strategies Suit High and Low Budgetscited by 131
  2. 2022Active Learning Through a Covering Lenscited by 62
  3. 2023A comprehensive survey on deep active learning in medical image analysiscited by 32
  4. 2022PT4AL: Using Self-Supervised Pretext Tasks for Active Learningcited by 27
  5. 2022Making Your First Choice: To Address Cold Start Problem in Vision Active Learningcited by 25
  6. 2024Revisiting Active Learning in the Era of Vision Foundation Modelscited by 13
  7. 2024Making Better Use of Unlabelled Data in Bayesian Active Learningcited by 10
  8. 2023BaSAL: Size-Balanced Warm Start Active Learning for LiDAR Semantic Segmentationcited by 8
  9. 2023You Never Get a Second Chance To Make a Good First Impression: Seeding Active Learning for 3D Semantic Segmentationcited by 6
  10. 2023NTKCPL: Active Learning on Top of Self-Supervised Model by Estimating True Coveragecited by 3
  11. 2024Leveraging image captions for selective whole slide image annotationcited by 1
  12. 2022Label-Efficient Interactive Time-Series Anomaly Detectioncited by 0
3 more
  1. 2023Taming Small-sample Bias in Low-budget Active Learningcited by 0
  2. 2023Pool-Based Active Learning with Proper Topological Regionscited by 0
  3. 2025Cold-Start Active Preference Learning in Socio-Economic Domainscited by 0