# SELC: Self-Ensemble Label Correction Improves Learning with Noisy Labels

arXiv: https://arxiv.org/abs/2205.01156 · HTML: https://mlsanity.com/paper/2205.01156/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 44 references that are in the map.
- First posted on arXiv 2 May 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [WebVision Database: Visual Learning and Understanding from Web Data](https://mlsanity.com/paper/1708.02862/) (2017)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Probabilistic End-to-end Noise Correction for Learning with Noisy Labels](https://mlsanity.com/paper/1903.07788/) (2019)
- [Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels](https://mlsanity.com/paper/1905.05040/) (2019)
- [SELF: Learning to Filter Noisy Labels with Self-Ensembling](https://mlsanity.com/paper/1910.01842/) (2019)
- [NLNL: Negative Learning for Noisy Labels](https://mlsanity.com/paper/1908.07387/) (2019)
- [Iterative Learning with Open-set Noisy Labels](https://mlsanity.com/paper/1804.00092/) (2018)
- [Learning with Feature-Dependent Label Noise: A Progressive Approach](https://mlsanity.com/paper/2103.07756/) (2021)
- [Combating Label Noise in Deep Learning Using Abstention](https://mlsanity.com/paper/1905.10964/) (2019)
- [Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise](https://mlsanity.com/paper/2012.05458/) (2020)
- [Error-Bounded Correction of Noisy Labels](https://mlsanity.com/paper/2011.10077/) (2020)
- [Towards Understanding Deep Learning from Noisy Labels with Small-Loss Criterion](https://mlsanity.com/paper/2106.09291/) (2021)
- [Boosting Co-teaching with Compression Regularization for Label Noise](https://mlsanity.com/paper/2104.13766/) (2021)
- [Coresets for Robust Training of Neural Networks against Noisy Labels](https://mlsanity.com/paper/2011.07451/) (2020)

## Built on by (most cited first, 24 of 24)

- [ProMix: Combating Label Noise via Maximizing Clean Sample Utility](https://mlsanity.com/paper/2207.10276/) (2022)
- [Early Stopping Against Label Noise Without Validation Data](https://mlsanity.com/paper/2502.07551/) (2025)
- [Practical Edge Detection via Robust Collaborative Learning](https://mlsanity.com/paper/2308.14084/) (2023)
- [CrossSplit: Mitigating Label Noise Memorization through Data Splitting](https://mlsanity.com/paper/2212.01674/) (2022)
- [Mitigating Label Noise through Data Ambiguation](https://mlsanity.com/paper/2305.13764/) (2023)
- [Foster Adaptivity and Balance in Learning with Noisy Labels](https://mlsanity.com/paper/2407.02778/) (2024)
- [Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selection](https://mlsanity.com/paper/2402.11242/) (2024)
- [Compensation Learning in Semantic Segmentation](https://mlsanity.com/paper/2304.13428/) (2023)
- [Differences Between Hard and Noisy-labeled Samples: An Empirical Study](https://mlsanity.com/paper/2307.10718/) (2023)
- [Leveraging Unlabeled Data to Track Memorization](https://mlsanity.com/paper/2212.04461/) (2022)
- [May the Forgetting Be with You: Alternate Replay for Learning with Noisy Labels](https://mlsanity.com/paper/2408.14284/) (2024)
- [Efficient Adaptive Label Refinement for Label Noise Learning](https://mlsanity.com/paper/2502.00386/) (2025)
- [Neighbour Consistency Guided Pseudo-Label Refinement for Unsupervised Person Re-Identification](https://mlsanity.com/paper/2211.16847/) (2022)
- [Correcting Noisy Multilabel Predictions: Modeling Label Noise through Latent Space Shifts](https://mlsanity.com/paper/2502.14281/) (2025)
- [On Robust Learning from Noisy Labels: A Permutation Layer Approach](https://mlsanity.com/paper/2211.15890/) (2022)
- [Group Benefits Instances Selection for Data Purification](https://mlsanity.com/paper/2403.15694/) (2024)
- [Mitigating Noisy Supervision Using Synthetic Samples with Soft Labels](https://mlsanity.com/paper/2406.16966/) (2024)
- [SSP-RACL: Classification of Noisy Fundus Images with Self-Supervised Pretraining and Robust Adaptive Credal Loss](https://mlsanity.com/paper/2409.18147/) (2024)
- [Dynamical Label Augmentation and Calibration for Noisy Electronic Health Records](https://mlsanity.com/paper/2505.07320/) (2025)
- [PEdger++: Practical Edge Detection via Assembling Cross Information](https://mlsanity.com/paper/2508.11961/) (2025)
- [Noise-aware few-shot learning through bi-directional multi-view prompt alignment](https://mlsanity.com/paper/2603.11617/) (2026)
- [Noise-Aware Framework for Correcting Corrupted Labels](https://mlsanity.com/paper/2606.11695/) (2026)
- [XMix: Combating Extremely Noisy Labels via Local Smoothness in Self-Supervised Feature Space](https://mlsanity.com/paper/2607.23865/) (2026)
- [PaSta: Noisy Node Classification with Partial Label Learning](https://mlsanity.com/paper/2608.25365/) (2026)

## Abstract (arXiv)

Deep neural networks are prone to overfitting noisy labels, resulting in poor generalization performance. To overcome this problem, we present a simple and effective method self-ensemble label correction (SELC) to progressively correct noisy labels and refine the model. We look deeper into the memorization behavior in training with noisy labels and observe that the network outputs are reliable in the early stage. To retain this reliable knowledge, SELC uses ensemble predictions formed by an exponential moving average of network outputs to update the original noisy labels. We show that training with SELC refines the model by gradually reducing supervision from noisy labels and increasing supervision from ensemble predictions. Despite its simplicity, compared with many state-of-the-art methods, SELC obtains more promising and stable results in the presence of class-conditional, instance-dependent, and real-world label noise. The code is available at https://github.com/MacLLL/SELC.