mlsanityData to 7 Oct 2026

On Statistical Bias In Active Learning: How and When To Fix It

arXiv 2101.11665 ↗First posted 27 Jan 2021stat.ML cs.LGcited by 72

Not on the map. Why?

Abstract (arXiv)

Active learning is a powerful tool when labelling data is expensive, but it introduces a bias because the training data no longer follows the population distribution. We formalize this bias and investigate the situations in which it can be harmful and sometimes even helpful. We further introduce novel corrective weights to remove bias when doing so is beneficial. Through this, our work not only provides a useful mechanism that can improve the active learning approach, but also an explanation of the empirical successes of various existing approaches which ignore this bias. In particular, we show that this bias can be actively helpful when training overparameterized models -- like neural networks -- with relatively little data.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Builds on 25 of its 55 references that are in the map. First posted on arXiv 27 Jan 2021.

Builds on

25 of 55 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2019On the Convergence of Adam and Beyondcited by 834
  4. 2017Deep Bayesian Active Learning with Image Datacited by 812
  5. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  6. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  7. 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 337
  8. 2019Learning Loss for Active Learningcited by 335
  9. 2019Variational Adversarial Active Learningcited by 284
  10. 2018What is the Effect of Importance Weighting in Deep Learning?cited by 250
  11. 2017Concrete Dropoutcited by 227
  12. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 218
13 more
  1. 2017Cost-Effective Active Learning for Deep Image Classificationcited by 165
  2. 2017Deep Active Learning for Named Entity Recognitioncited by 108
  3. 2018Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Studycited by 90
  4. 2018Practical Obstacles to Deploying Active Learningcited by 68
  5. 2017Active Learning for Cost-Sensitive Classificationcited by 40
  6. 2018Active Learning with Partial Feedbackcited by 27
  7. 2019Half a Percent of Labels is Enough: Efficient Animal Detection in UAV Imagery using Deep CNNs and Active Learningcited by 22
  8. 2019Bayesian semi-supervised learning for uncertainty-calibrated prediction of molecular properties and active learningcited by 21
  9. 2018Cost-Effective Training of Deep CNNs with Active Model Adaptationcited by 16
  10. 2019Active Learning for Decision-Making from Imbalanced Observational Datacited by 16
  11. 2018Active Testing: An Efficient and Robust Framework for Estimating Accuracycited by 15
  12. 2018Leveraging Crowdsourcing Data For Deep Active Learning - An Application: Learning Intents in Alexacited by 11
  13. 2018Active Learning with Logged Datacited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 17; 2023: 16; 2024: 8; 2025: 9; 2026: 8.

72 papers.

Sort:

By area:

  1. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 233
  2. 2023Modern Bayesian Experimental Designcited by 121
  3. 2021DEUP: Direct Epistemic Uncertainty Predictioncited by 102
  4. 2023Anchor Points: Benchmarking Models with Much Fewer Examplescited by 84
  5. 2021Active Testing: Sample-Efficient Model Evaluationcited by 82
  6. 2024A Survey on Deep Active Learning: Recent Advances and New Frontierscited by 53
  7. 2022A Framework and Benchmark for Deep Batch Active Learning for Regressioncited by 47
  8. 2021Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learningcited by 36
  9. 2021Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Datacited by 27
  10. 2022Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluationcited by 26
  11. 2022A Lagrangian Duality Approach to Active Learningcited by 24
  12. 2021Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outlierscited by 22
38 more
  1. 2022General Intelligence Requires Rethinking Explorationcited by 20
  2. 2021Class-Balanced Active Learning for Image Classificationcited by 15
  3. 2024A Short Survey on Importance Weighting for Machine Learningcited by 15
  4. 2021Active Learning in Bayesian Neural Networks with Balanced Entropy Learning Principlecited by 14
  5. 2022Breaking Feedback Loops in Recommender Systems with Causal Inferencecited by 14
  6. 2023Controllable Textual Inversion for Personalized Text-to-Image Generationcited by 14
  7. 2024A Framework for Efficient Model Evaluation through Stratification, Sampling, and Estimationcited by 13
  8. 2023D-CALM: A Dynamic Clustering-based Active Learning Approach for Mitigating Biascited by 12
  9. 2025Step-DAD: Semi-Amortized Policy-Based Bayesian Experimental Designcited by 12
  10. 2025Scaling Up Active Testing to Large Language Modelscited by 12
  11. 2023Gaussian Switch Sampling: A Second Order Approach to Active Learningcited by 10
  12. 2022Characterizing the robustness of Bayesian adaptive experimental designs to active learning biascited by 8
  13. 2021Prioritized training on points that are learnable, worth learning, and not yet learned (workshop version)cited by 7
  14. 2023Metrics for Bayesian Optimal Experiment Design under Model Misspecificationcited by 7
  15. 2024Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitionscited by 7
  16. 2025Consensus-Driven Active Model Selectioncited by 7
  17. 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safetycited by 7
  18. 2021Mitigating Sampling Bias and Improving Robustness in Active Learningcited by 5
  19. 2022TrustAL: Trustworthy Active Learning using Knowledge Distillationcited by 5
  20. 2022Integrating Reward Maximization and Population Estimation: Sequential Decision-Making for Internal Revenue Service Audit Selectioncited by 5
  21. 2023Robust online active learningcited by 5
  22. 2023Bayesian Active Learning in the Presence of Nuisance Parameterscited by 5
  23. 2024Bayesian Active Learning for Semantic Segmentationcited by 5
  24. 2023Primal Dual Continual Learning: Balancing Stability and Plasticity through Adaptive Memory Allocationcited by 4
  25. 2023Active Transfer Learning for Efficient Video-Specific Human Pose Estimationcited by 4
  26. 2025Active Measurement: Efficient Estimation at Scalecited by 4
  27. 2026Active Testing of Large Language Models via Approximate Neyman Allocationcited by 4
  28. 2021Addressing Bias in Active Learning with Depth Uncertainty Networks... or Notcited by 3
  29. 2022Marginal and Joint Cross-Entropies & Predictives for Online Bayesian Inference, Active Learning, and Active Samplingcited by 3
  30. 2022Is More Data All You Need? A Causal Explorationcited by 3
  31. 2022Pareto Optimization for Active Learning under Out-of-Distribution Data Scenarioscited by 3
  32. 2022Margin-based sampling in high dimensions: When being active is less efficient than staying passivecited by 3
  33. 2023On Dataset Transferability in Active Learning for Transformerscited by 3
  34. 2023ActiveDC: Distribution Calibration for Active Finetuningcited by 3
  35. 2024Downstream-Pretext Domain Knowledge Traceback for Active Learningcited by 3
  36. 2021Robust Contrastive Active Learning with Feature-guided Query Strategiescited by 2
  37. 2021Depth Uncertainty Networks for Active Learningcited by 2
  38. 2023Pushing the Accuracy-Group Robustness Frontier with Introspective Self-playcited by 2

50 most cited of 72 All 72 (JSON)