# ProSelfLC: Progressive Self Label Correction Towards A Low-Temperature Entropy State arXiv: https://arxiv.org/abs/2207.00118 ยท HTML: https://mlsanity.com/paper/2207.00118/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Builds on 69 of its 115 references that are in the map. - First posted on arXiv 30 Jun 2022. ## Builds on (most cited first, 30 of 69) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015) - [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017) - [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017) - [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016) - [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014) - [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018) - [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018) - [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018) - [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019) - [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017) - [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020) - [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018) - [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016) - [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017) - [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017) - [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018) - [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019) - [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019) - [Do Deep Nets Really Need to be Deep?](https://mlsanity.com/paper/1312.6184/) (2013) - [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017) - [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020) - [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019) - [Revisiting the Calibration of Modern Neural Networks](https://mlsanity.com/paper/2106.07998/) (2021) - [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019) - [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014) - [Verified Uncertainty Calibration](https://mlsanity.com/paper/1909.10155/) (2019) - [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018) ## Built on by (most cited first, 5 of 5) - [Semi-Supervised Domain Adaptation with Source Label Adaptation](https://mlsanity.com/paper/2302.02335/) (2023) - [Survey on Leveraging Uncertainty Estimation Towards Trustworthy Deep Neural Networks: The Case of Reject Option and Post-training Processing](https://mlsanity.com/paper/2304.04906/) (2023) - [How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise in Machine Translation](https://mlsanity.com/paper/2407.02208/) (2024) - [Enhancing Sample Selection Against Label Noise by Cutting Mislabeled Easy Examples](https://mlsanity.com/paper/2502.08227/) (2025) - [Efficient Content-based Recommendation Model Training via Noise-aware Coreset Selection](https://mlsanity.com/paper/2601.10067/) (2026) ## Abstract (arXiv) There is a family of label modification approaches including self and non-self label correction (LC), and output regularisation. They are widely used for training robust deep neural networks (DNNs), but have not been mathematically and thoroughly analysed together. We study them and discover three key issues: (1) We are more interested in adopting Self LC as it leverages its own knowledge and requires no auxiliary models. However, it is unclear how to adaptively trust a learner as the training proceeds. (2) Some methods penalise while the others reward low-entropy (i.e., high-confidence) predictions, prompting us to ask which one is better. (3) Using the standard training setting, a learned model becomes less confident when severe noise exists. Self LC using high-entropy knowledge would generate high-entropy targets. To resolve the issue (1), inspired by a well-accepted finding, i.e., deep neural networks learn meaningful patterns before fitting noise, we propose a novel end-to-end method named ProSelfLC, which is designed according to the learning time and prediction entropy. Concretely, for any data point, we progressively and adaptively trust its predicted probability distribution versus its annotated one if a network has been trained for a relatively long time and the prediction is of low entropy. For the issue (2), the effectiveness of ProSelfLC defends entropy minimisation. By ProSelfLC, we empirically prove that it is more effective to redefine a semantic low-entropy state and optimise the learner toward it. To address the issue (3), we decrease the entropy of self knowledge using a low temperature before exploiting it to correct labels, so that the revised labels redefine low-entropy target probability distributions. We demonstrate the effectiveness of ProSelfLC through extensive experiments in both clean and noisy settings, and on both image and protein datasets.