mlsanityData to 7 Oct 2026

Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets

arXiv 2202.02794 ↗First posted 6 Feb 2022cs.LGcited by 131

Not on the map. Why?

Abstract (arXiv)

Investigating active learning, we focus on the relation between the number of labeled examples (budget size), and suitable querying strategies. Our theoretical analysis shows a behavior reminiscent of phase transition: typical examples are best queried when the budget is low, while unrepresentative examples are best queried when the budget is large. Combined evidence shows that a similar phenomenon occurs in common classification models. Accordingly, we propose TypiClust -- a deep active learning strategy suited for low budgets. In a comparative empirical investigation of supervised learning, using a variety of architectures and image datasets, TypiClust outperforms all other active learning strategies in the low-budget regime. Using TypiClust in the semi-supervised framework, performance gets an even more significant boost. In particular, state-of-the-art semi-supervised methods trained on CIFAR-10 with 10 labeled examples selected by TypiClust, reach 93.2% accuracy -- an improvement of 39.4% over random selection. Code is available at https://github.com/avihu111/TypiClust.

Key facts. Cited by 131 papers in the map (data through 7 Oct 2026). Builds on 41 of its 75 references that are in the map. First posted on arXiv 6 Feb 2022.

Builds on

41 of 75 references.

References · 41

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  8. 2017Deep Bayesian Active Learning with Image Datacited by 812
  9. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  10. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  11. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
  12. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
29 more
  1. 2015An Introduction to Matrix Concentration Inequalitiescited by 391
  2. 2020SCAN: Learning to Classify Images without Labelscited by 351
  3. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  4. 2019Learning Loss for Active Learningcited by 335
  5. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 285
  6. 2019Variational Adversarial Active Learningcited by 284
  7. 2020Cold-start Active Learning through Self-supervised Language Modelingcited by 128
  8. 2019Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Costcited by 114
  9. 2019Discriminative Active Learningcited by 97
  10. 2019Deep Active Learning: Unified and Principled Method for Query and Trainingcited by 92
  11. 2019Diverse mini-batch Active Learningcited by 88
  12. 2017Deep Active Learning over the Long Tailcited by 82
  13. 2018Practical Obstacles to Deploying Active Learningcited by 68
  14. 2019Let's Agree to Agree: Neural Networks Share Classification Order on Real Datasetscited by 55
  15. 2020Towards Robust and Reproducible Active Learning Using Neural Networkscited by 53
  16. 2020Improving Unsupervised Image Clustering With Robust Learningcited by 51
  17. 2020A Survey of Active Learning for Text Classification using Deep Neural Networkscited by 50
  18. 2021Reducing Label Effort: Self-Supervised meets Active Learningcited by 42
  19. 2019Rethinking deep active learning: Using unlabeled data at model trainingcited by 39
  20. 2019Parting with Illusions about Deep Active Learningcited by 38
  21. 2021The Grammar-Learning Trajectories of Neural Language Modelscited by 36
  22. 2021Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approachcited by 34
  23. 2018Addressing the Item Cold-start Problem by Attribute-driven Active Learningcited by 33
  24. 2018Theory of Curriculum Learning, with Convex Loss Functionscited by 33
  25. 2020On the Marginal Benefit of Active Learning: Does Self-Supervision Eat Its Cake?cited by 18
  26. 2020On Initial Pools for Deep Active Learningcited by 18
  27. 2021A Simple Baseline for Low-Budget Active Learningcited by 15
  28. 2020Deep Active Learning with Augmentation-based Consistency Estimationcited by 8
  29. 2020Boosting the Performance of Semi-Supervised Learning with Unsupervised Clusteringcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 25; 2024: 38; 2025: 26; 2026: 31.

131 papers.

Sort:

By area:

  1. 2022Active Learning Through a Covering Lenscited by 62
  2. 2024A Survey on Deep Active Learning: Recent Advances and New Frontierscited by 53
  3. 2023A comprehensive survey on deep active learning in medical image analysiscited by 32
  4. 2022Making Your First Choice: To Address Cold Start Problem in Vision Active Learningcited by 25
  5. 2023Beyond Confidence: Reliable Models Should Also Consider Atypicalitycited by 25
  6. 2024Towards Explainable, Safe Autonomous Driving with Language Embeddings for Novelty Identification and Active Learning: Framework and Experimental Analysis with Real-World Data Setscited by 25
  7. 2023LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learningcited by 20
  8. 2025LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settingscited by 19
  9. 2023Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networkscited by 18
  10. 2022Cold-Start Data Selection for Few-shot Language Model Fine-tuning: A Prompt-Based Uncertainty Propagation Approachcited by 16
  11. 2023How to Select Which Active Learning Strategy is Best Suited for Your Specific Problem and Budgetcited by 16
  12. 2024Generalized Coverage for More Robust Low-Budget Active Learningcited by 15
38 more
  1. 2023ActiveGLAE: A Benchmark for Deep Active Learning with Transformerscited by 14
  2. 2024Revisiting Active Learning in the Era of Vision Foundation Modelscited by 13
  3. 2024Uncertainty Herding: One Active Learning Method for All Label Budgetscited by 12
  4. 2025Filter Images First, Generate Instructions Later: Pre-Instruction Data Selection for Visual Instruction Tuningcited by 12
  5. 2024A Cross-Domain Benchmark for Active Learningcited by 11
  6. 2022Is margin all you need? An extensive empirical study of active learning on tabular datacited by 10
  7. 2024Computer Vision for Primate Behavior Analysis in the Wildcited by 10
  8. 2024Making Better Use of Unlabelled Data in Bayesian Active Learningcited by 10
  9. 2024Towards Deep Active Learning in Avian Bioacousticscited by 10
  10. 2023On the Limitations of Simulating Active Learningcited by 9
  11. 2023Towards Balanced Active Learning for Multimodal Classificationcited by 9
  12. 2023COLosSAL: A Benchmark for Cold-start Active Learning for 3D Medical Image Segmentationcited by 9
  13. 2023BAL: Balancing Diversity and Novelty for Active Learningcited by 9
  14. 2024ELFS: Label-Free Coreset Selection with Proxy Training Dynamicscited by 9
  15. 2023BaSAL: Size-Balanced Warm Start Active Learning for LiDAR Semantic Segmentationcited by 8
  16. 2024A Unified Approach Towards Active Learning and Out-of-Distribution Detectioncited by 7
  17. 2024Predicting the Susceptibility of Examples to Catastrophic Forgettingcited by 7
  18. 2024Integrating Deep Metric Learning with Coreset for Active Learning in 3D Segmentationcited by 7
  19. 2025Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generationcited by 7
  20. 2023You Never Get a Second Chance To Make a Good First Impression: Seeding Active Learning for 3D Semantic Segmentationcited by 6
  21. 2024Bridging Diversity and Uncertainty in Active learning with Self-Supervised Pre-Trainingcited by 6
  22. 2024Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classificationcited by 6
  23. 2026VisNec: Measuring and Leveraging Visual Necessity for Multimodal Instruction Tuningcited by 6
  24. 2024ALPBench: A Benchmark for Active Learning Pipelines on Tabular Datacited by 5
  25. 2024Bayesian Active Learning for Semantic Segmentationcited by 5
  26. 2024Deep Active Learning in the Open Worldcited by 5
  27. 2024Active Learning via Classifier Impact and Greedy Selection for Interactive Image Retrievalcited by 5
  28. 2025Reducing Aleatoric and Epistemic Uncertainty through Multi-modal Data Acquisitioncited by 5
  29. 2022Active Self-Semi-Supervised Learning for Few Labeled Samplescited by 4
  30. 2023Primal Dual Continual Learning: Balancing Stability and Plasticity through Adaptive Memory Allocationcited by 4
  31. 2023Re-weighting Tokens: A Simple and Effective Active Learning Strategy for Named Entity Recognitioncited by 4
  32. 2024Feature Alignment: Rethinking Efficient Active Learning via Proxy in the Context of Pre-trained Modelscited by 4
  33. 2022Margin-based sampling in high dimensions: When being active is less efficient than staying passivecited by 3
  34. 2023ASPEST: Bridging the Gap Between Active Learning and Selective Predictioncited by 3
  35. 2023NTKCPL: Active Learning on Top of Self-Supervised Model by Estimating True Coveragecited by 3
  36. 2023Learning to Rank for Active Learning via Multi-Task Bilevel Optimizationcited by 3
  37. 2023ActiveDC: Distribution Calibration for Active Finetuningcited by 3
  38. 2024Understanding Uncertainty-based Active Learning Under Model Mismatchcited by 3

50 most cited of 131 All 131 (JSON)