# Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection

arXiv: https://arxiv.org/abs/2111.14932 · HTML: https://mlsanity.com/paper/2111.14932/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 84 of its 123 references that are in the map.
- First posted on arXiv 29 Nov 2021.

## Builds on (most cited first, 30 of 84)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks](https://mlsanity.com/paper/1707.09564/) (2017)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017)
- [Identifying Mislabeled Data using the Area Under the Margin Ranking](https://mlsanity.com/paper/2001.10528/) (2020)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations](https://mlsanity.com/paper/2110.12088/) (2021)

## Built on by (most cited first, 20 of 20)

- [Instance-dependent Label Distribution Estimation for Learning with Label Noise](https://mlsanity.com/paper/2212.08380/) (2022)
- [Mitigating Label Noise through Data Ambiguation](https://mlsanity.com/paper/2305.13764/) (2023)
- [Regularly Truncated M-estimators for Learning with Noisy Labels](https://mlsanity.com/paper/2309.00894/) (2023)
- [Enhanced Meta Label Correction for Coping with Label Corruption](https://mlsanity.com/paper/2305.12961/) (2023)
- [SelectMix: Enhancing Label Noise Robustness through Targeted Sample Mixing](https://mlsanity.com/paper/2509.11265/) (2025)
- [Instance-dependent Noisy-label Learning with Graphical Model Based Noise-rate Estimation](https://mlsanity.com/paper/2305.19486/) (2023)
- [Meta-Learning Guided Label Noise Distillation for Robust Signal Modulation Classification](https://mlsanity.com/paper/2408.05151/) (2024)
- [Sample Selection via Contrastive Fragmentation for Noisy Label Regression](https://mlsanity.com/paper/2502.17771/) (2025)
- [Curvature Learning for Generalization of Hyperbolic Neural Networks](https://mlsanity.com/paper/2508.17232/) (2025)
- [Guiding Noisy Label Conditional Diffusion Models with Score-based Discriminator Correction](https://mlsanity.com/paper/2508.19581/) (2025)
- [LNL+K: Enhancing Learning with Noisy Labels Through Noise Source Knowledge Integration](https://mlsanity.com/paper/2306.11911/) (2023)
- [Noise-Aware Generalization: Robustness to In-Domain Noise and Out-of-Domain Generalization](https://mlsanity.com/paper/2504.02996/) (2025)
- [Robust Learning of Diffusion Models with Extremely Noisy Conditions](https://mlsanity.com/paper/2510.10149/) (2025)
- [A Unified Framework for Connecting Noise Modeling to Boost Noise Detection](https://mlsanity.com/paper/2312.00827/) (2023)
- [In-Context Learning with Noisy Labels](https://mlsanity.com/paper/2411.19581/) (2024)
- [Set a Thief to Catch a Thief: Combating Label Noise through Noisy Meta Learning](https://mlsanity.com/paper/2502.16104/) (2025)
- [MANTRA: a Framework for Multi-stage Adaptive Noise TReAtment During Training](https://mlsanity.com/paper/2512.04319/) (2025)
- [Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels](https://mlsanity.com/paper/2603.13894/) (2026)
- [Variational Rectification Inference for Learning with Noisy Labels](https://mlsanity.com/paper/2603.17255/) (2026)
- [SLT: Robust Quantum Neural Networks for Noisy-Label Medical Image Classification via Supermartingale-based Label Transition](https://mlsanity.com/paper/2607.16293/) (2026)

## Abstract (arXiv)

Recent studies on learning with noisy labels have shown remarkable performance by exploiting a small clean dataset. In particular, model agnostic meta-learning-based label correction methods further improve performance by correcting noisy labels on the fly. However, there is no safeguard on the label miscorrection, resulting in unavoidable performance degradation. Moreover, every training step requires at least three back-propagations, significantly slowing down the training speed. To mitigate these issues, we propose a robust and efficient method that learns a label transition matrix on the fly. Employing the transition matrix makes the classifier skeptical about all the corrected samples, which alleviates the miscorrection issue. We also introduce a two-head architecture to efficiently estimate the label transition matrix every iteration within a single back-propagation, so that the estimated matrix closely follows the shifting noise distribution induced by label correction. Extensive experiments demonstrate that our approach shows the best performance in training efficiency while having comparable or better accuracy than existing methods.