mlsanityData to 7 Oct 2026

Towards Robust and Reproducible Active Learning Using Neural Networks

arXiv 2002.09564 ↗First posted 21 Feb 2020cs.LG cs.CV stat.MLcited by 53

Not on the map. Why?

Abstract (arXiv)

Active learning (AL) is a promising ML paradigm that has the potential to parse through large unlabeled data and help reduce annotation cost in domains where labeling data can be prohibitive. Recently proposed neural network based AL methods use different heuristics to accomplish this goal. In this study, we demonstrate that under identical experimental settings, different types of AL algorithms (uncertainty based, diversity based, and committee based) produce an inconsistent gain over random sampling baseline. Through a variety of experiments, controlling for sources of stochasticity, we show that variance in performance metrics achieved by AL algorithms can lead to results that are not consistent with the previously reported results. We also found that under strong regularization, AL methods show marginal or no advantage over the random sampling baseline under a variety of experimental conditions. Finally, we conclude with a set of recommendations on how to assess the results using a new AL algorithm to ensure results are reproducible and robust under changes in experimental conditions. We share our codes to facilitate AL evaluations. We believe our findings and recommendations will help advance reproducible research in AL using neural networks. We open source our code at https://github.com/PrateekMunjal/TorchAL

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 6; 2023: 20; 2024: 9; 2025: 6; 2026: 8.

53 papers.

Sort:

By area:

  1. 2022Active Learning on a Budget: Opposite Strategies Suit High and Low Budgetscited by 131
  2. 2022A Comparative Survey of Deep Active Learningcited by 88
  3. 2022Active Learning Through a Covering Lenscited by 62
  4. 2023Prediction-Oriented Bayesian Active Learningcited by 60
  5. 2023AbdomenAtlas-8K: Annotating 8,000 CT Volumes for Multi-Organ Segmentation in Three Weekscited by 52
  6. 2021Effective Evaluation of Deep Active Learning on Image Classification Taskscited by 37
  7. 2023Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessmentcited by 25
  8. 2023Active Prompt Learning in Vision Language Modelscited by 25
  9. 2023Active learning for medical image segmentation with stochastic batchescited by 20
  10. 2023How to Select Which Active Learning Strategy is Best Suited for Your Specific Problem and Budgetcited by 16
  11. 2023ActiveGLAE: A Benchmark for Deep Active Learning with Transformerscited by 14
  12. 2023Box-Level Active Detectioncited by 13
38 more
  1. 2024Revisiting Active Learning in the Era of Vision Foundation Modelscited by 13
  2. 2024A Cross-Domain Benchmark for Active Learningcited by 11
  3. 2022Is margin all you need? An extensive empirical study of active learning on tabular datacited by 10
  4. 2023Best Practices in Active Learning for Semantic Segmentationcited by 10
  5. 2021Learning to Limit Data Collection via Scaling Laws: A Computational Interpretation for the Legal Principle of Data Minimizationcited by 9
  6. 2024Integrating Deep Metric Learning with Coreset for Active Learning in 3D Segmentationcited by 7
  7. 2021Efficacy of Bayesian Neural Networks in Active Learningcited by 6
  8. 2022An Empirical Study on the Efficacy of Deep Active Learning for Image Classificationcited by 6
  9. 2024A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentationcited by 6
  10. 2021A Survey on Active Deep Learning: From Model-driven to Data-drivencited by 4
  11. 2023A Unified Active Learning Framework for Annotating Graph Data with Application to Software Source Code Performance Predictioncited by 4
  12. 2023An Expanded Benchmark that Rediscovers and Affirms the Edge of Uncertainty Sampling for Active Learning in Tabular Datasetscited by 4
  13. 2024Feasibility Study on Active Learning of Smart Surrogates for Scientific Simulationscited by 4
  14. 2023NTKCPL: Active Learning on Top of Self-Supervised Model by Estimating True Coveragecited by 3
  15. 2024Understanding Uncertainty-based Active Learning Under Model Mismatchcited by 3
  16. 2025Active Learning with a Noisy Annotatorcited by 3
  17. 2022Efficient Bayesian Updates for Deep Active Learning via Laplace Approximationscited by 2
  18. 2023Towards Label-Efficient Incremental Learning: A Surveycited by 2
  19. 2023Training Ensembles with Inliers and Outliers for Semi-supervised Active Learningcited by 2
  20. 2023How To Overcome Confirmation Bias in Semi-Supervised Image Classification By Active Learningcited by 2
  21. 2024DCoM: Active Learning for All Learnerscited by 2
  22. 2025To Label or Not to Label: PALM -- A Predictive Model for Evaluating Sample Efficiency in Active Learning Modelscited by 2
  23. 2025Cleaning the Pool: Progressive Filtering of Unlabeled Pools in Deep Active Learningcited by 2
  24. 2023OpenAL: Evaluation and Interpretation of Active Learning Strategiescited by 1
  25. 2023Transfer and Active Learning for Dissonance Detection: Addressing the Rare-Class Challengecited by 1
  26. 2023IAdet: Simplest human-in-the-loop object detectioncited by 1
  27. 2025Cream of the Crop: Harvesting Rich, Scalable and Transferable Multi-Modal Data for Instruction Fine-Tuningcited by 1
  28. 2025No Free Lunch in Active Learning: LLM Embedding Quality Dictates Query Strategy Successcited by 1
  29. 2026Finally Outshining the Random Baseline: A Simple and Effective Solution for Active Learning in 3D Biomedical Imagingcited by 1
  30. 2026The Speed-up Factor: A Quantitative Multi-Iteration Active Learning Performance Metriccited by 1
  31. 2026BoSS: A Best-of-Strategies Selector as an Oracle for Deep Active Learningcited by 1
  32. 2026BioDCASE: Active Learning for Bioacousticscited by 1
  33. 2023Active Vision Reinforcement Learning under Limited Visual Observabilitycited by 0
  34. 2023Towards Comparable Active Learningcited by 0
  35. 2024Diverse Subset Selection via Norm-Based Sampling and Orthogonalitycited by 0
  36. 2024Combining X-Vectors and Bayesian Batch Active Learning: Two-Stage Active Learning Pipeline for Speech Recognitioncited by 0
  37. 2025Cost-effective Reduced-Order Modeling via Bayesian Active Learningcited by 0
  38. 2026Not All Forgetting Is Equal: Retention Dynamics in Fine-Tuned Image Classifierscited by 0

50 most cited of 53 All 53 (JSON)