# Adaptive Sample Selection for Robust Learning under Label Noise

arXiv: https://arxiv.org/abs/2106.15292 · HTML: https://mlsanity.com/paper/2106.15292/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 59 references that are in the map.
- First posted on arXiv 29 Jun 2021.

## Builds on (most cited first, 30 of 38)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Decoupling "when to update" from "how to update"](https://mlsanity.com/paper/1706.02613/) (2017)
- [Probabilistic End-to-end Noise Correction for Learning with Noisy Labels](https://mlsanity.com/paper/1903.07788/) (2019)
- [CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise](https://mlsanity.com/paper/1711.07131/) (2017)
- [Dimensionality-Driven Learning with Noisy Labels](https://mlsanity.com/paper/1806.02612/) (2018)
- [Learning to Learn from Noisy Labeled Data](https://mlsanity.com/paper/1812.05214/) (2018)
- [Learning with Feature-Dependent Label Noise: A Progressive Approach](https://mlsanity.com/paper/2103.07756/) (2021)
- [A Second-Order Approach to Learning with Instance-Dependent Label Noise](https://mlsanity.com/paper/2012.11854/) (2020)
- [Curriculum Loss: Robust Learning and Generalization against Label Corruption](https://mlsanity.com/paper/1905.10045/) (2019)

## Built on by (most cited first, 21 of 21)

- [Adaptive Integration of Partial Label Learning and Negative Learning for Enhanced Noisy Label Learning](https://mlsanity.com/paper/2312.09505/) (2023)
- [When Noisy Labels Meet Long Tail Dilemmas: A Representation Calibration Method](https://mlsanity.com/paper/2211.10955/) (2022)
- [Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels](https://mlsanity.com/paper/2305.19518/) (2023)
- [CTRL: Clustering Training Losses for Label Error Detection](https://mlsanity.com/paper/2208.08464/) (2022)
- [ERASE: Error-Resilient Representation Learning on Graphs for Label Noise Tolerance](https://mlsanity.com/paper/2312.08852/) (2023)
- [SAP: Corrective Machine Unlearning with Scaled Activation Projection for Label Noise Robustness](https://mlsanity.com/paper/2403.08618/) (2024)
- [Diversified and Personalized Multi-rater Medical Image Segmentation](https://mlsanity.com/paper/2403.13417/) (2024)
- [NLPrompt: Noise-Label Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2412.01256/) (2024)
- [Regularly Truncated M-estimators for Learning with Noisy Labels](https://mlsanity.com/paper/2309.00894/) (2023)
- [Skeleton-Based Human Action Recognition with Noisy Labels](https://mlsanity.com/paper/2403.09975/) (2024)
- [Underwater Object Detection in the Era of Artificial Intelligence: Current, Challenge, and Future](https://mlsanity.com/paper/2410.05577/) (2024)
- [Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selection](https://mlsanity.com/paper/2402.11242/) (2024)
- [Is the Information Bottleneck Robust Enough? Towards Label-Noise Resistant Information Bottleneck Learning](https://mlsanity.com/paper/2512.10573/) (2025)
- [Balance-aware Sequence Sampling Makes Multi-modal Learning Better](https://mlsanity.com/paper/2501.01470/) (2025)
- [Group Benefits Instances Selection for Data Purification](https://mlsanity.com/paper/2403.15694/) (2024)
- [Coordinated Sparse Recovery of Label Noise](https://mlsanity.com/paper/2404.04800/) (2024)
- [SSP-RACL: Classification of Noisy Fundus Images with Self-Supervised Pretraining and Robust Adaptive Credal Loss](https://mlsanity.com/paper/2409.18147/) (2024)
- [Human-Corrected Labels Learning: Enhancing Labels Quality via Human Correction of VLMs Discrepancies](https://mlsanity.com/paper/2511.09063/) (2025)
- [Learning to Clean: Reinforcement Learning for Noisy Label Correction](https://mlsanity.com/paper/2511.19808/) (2025)
- [Intrinsic Gradient Suppression for Label-Noise Prompt Tuning in Vision-Language Models](https://mlsanity.com/paper/2605.00591/) (2026)
- [Prompt and Refinement: Asymmetric Mutual Learning for Infrared Small Target Detection with Noisy Labels](https://mlsanity.com/paper/2610.05918/) (2026)

## Abstract (arXiv)

Deep Neural Networks (DNNs) have been shown to be susceptible to memorization or overfitting in the presence of noisily-labelled data. For the problem of robust learning under such noisy data, several algorithms have been proposed. A prominent class of algorithms rely on sample selection strategies wherein, essentially, a fraction of samples with loss values below a certain threshold are selected for training. These algorithms are sensitive to such thresholds, and it is difficult to fix or learn these thresholds. Often, these algorithms also require information such as label noise rates which are typically unavailable in practice. In this paper, we propose an adaptive sample selection strategy that relies only on batch statistics of a given mini-batch to provide robustness against label noise. The algorithm does not have any additional hyperparameters for sample selection, does not need any information on noise rates and does not need access to separate data with clean labels. We empirically demonstrate the effectiveness of our algorithm on benchmark datasets.