mlsanityData to 7 Oct 2026

Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples

arXiv 1704.07433 ↗First posted 24 Apr 2017stat.ML cs.LGcited by 147

Not on the map. Why?

Abstract (arXiv)

Self-paced learning and hard example mining re-weight training instances to improve learning accuracy. This paper presents two improved alternatives based on lightweight estimates of sample uncertainty in stochastic gradient descent (SGD): the variance in predicted probability of the correct class across iterations of mini-batch SGD, and the proximity of the correct class probability to the decision threshold. Extensive experimental results on six datasets show that our methods reliably improve accuracy in various network architectures, including additional gains on top of other popular training techniques, such as residual learning, momentum, ADAM, batch normalization, dropout, and distillation.

Key facts. Cited by 147 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 39; 2022: 33; 2023: 34; 2024: 25; 2025: 7; 2026: 9.

147 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 288
  3. 2021Curriculum Learning: A Surveycited by 257
  4. 2024Rho-1: Not All Tokens Are What You Needcited by 132
  5. 2021Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspectivecited by 116
  6. 2022Coverage-centric Coreset Selection for High Pruning Ratescited by 115
  7. 2022Robust Training under Label Noise by Over-parameterizationcited by 93
  8. 2021Jo-SRC: A Contrastive Approach for Combating Noisy Labelscited by 80
  9. 2021Robust Audio-Visual Instance Discriminationcited by 80
  10. 2021Sample Selection with Uncertainty of Losses for Learning with Noisy Labelscited by 79
  11. 2022UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learningcited by 77
  12. 2021Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inferencecited by 66
38 more
  1. 2023Large-scale Dataset Pruning with Dynamic Uncertaintycited by 63
  2. 2021Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answeringcited by 62
  3. 2022Part-based Pseudo Label Refinement for Unsupervised Person Re-identificationcited by 59
  4. 2021Self-Damaging Contrastive Learningcited by 48
  5. 2021Sample Selection for Fair and Robust Trainingcited by 48
  6. 2022Divert More Attention to Vision-Language Trackingcited by 47
  7. 2021Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classificationcited by 43
  8. 2021On Tilted Losses in Machine Learning: Theory and Applicationscited by 42
  9. 2023Data Valuation Without Training of a Modelcited by 35
  10. 2021Learning from Noisy Labels for Entity-Centric Information Extractioncited by 34
  11. 2022Instance-specific and Model-adaptive Supervision for Semi-supervised Semantic Segmentationcited by 30
  12. 2022To Aggregate or Not? Learning with Separate Noisy Labelscited by 29
  13. 2023Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understandingcited by 28
  14. 2021Meta-learning with an Adaptive Task Schedulercited by 27
  15. 2022Towards Open Set Video Anomaly Detectioncited by 26
  16. 2022Making Your First Choice: To Address Cold Start Problem in Vision Active Learningcited by 25
  17. 2021Procrustean Training for Imbalanced Deep Learningcited by 24
  18. 2022CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learningcited by 24
  19. 2022Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Trainingcited by 24
  20. 2021Learning with Noisy Labels for Robust Point Cloud Segmentationcited by 22
  21. 2022The Two Dimensions of Worst-case Training and the Integrated Effect for Out-of-domain Generalizationcited by 21
  22. 2023Why Is Prompt Tuning for Vision-Language Models Robust to Noisy Labels?cited by 21
  23. 2021Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrectioncited by 20
  24. 2022Selective Prediction via Training Dynamicscited by 20
  25. 2021Deep Reinforcement Learning with Quantum-inspired Experience Replaycited by 18
  26. 2021Two Wrongs Don't Make a Right: Combating Confirmation Bias in Learning with Label Noisecited by 18
  27. 2021Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translationcited by 17
  28. 2021Accelerating Neural Architecture Search via Proxy Datacited by 14
  29. 2021Self-Diagnosing GAN: Diagnosing Underrepresented Samples in Generative Adversarial Networkscited by 13
  30. 2022FIFA: Making Fairness More Generalizable in Classifiers Trained on Imbalanced Datacited by 13
  31. 2023Hundreds Guide Millions: Adaptive Offline Reinforcement Learning with Expert Guidancecited by 13
  32. 2022Boosting Facial Expression Recognition by A Semi-Supervised Progressive Teachercited by 12
  33. 2023Curricular Object Manipulation in LiDAR-based Object Detectioncited by 12
  34. 2022TiDAL: Learning Training Dynamics for Active Learningcited by 11
  35. 2022Simplicity Bias Leads to Amplified Performance Disparitiescited by 11
  36. 2023Divert More Attention to Vision-Language Object Trackingcited by 11
  37. 2023Black-box Unsupervised Domain Adaptation with Bi-directional Atkinson-Shiffrin Memorycited by 11
  38. 2023Learning with Noisy labels via Self-supervised Adversarial Noisy Maskingcited by 10

50 most cited of 147 All 147 (JSON)