# Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization

arXiv: https://arxiv.org/abs/2102.02414 · HTML: https://mlsanity.com/paper/2102.02414/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 76 references that are in the map.
- First posted on arXiv 4 Feb 2021.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [Consistent Estimators for Learning to Defer to an Expert](https://mlsanity.com/paper/2006.01862/) (2020)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Decoupling "when to update" from "how to update"](https://mlsanity.com/paper/1706.02613/) (2017)
- [Are Anchor Points Really Indispensable in Label-Noise Learning?](https://mlsanity.com/paper/1906.00189/) (2019)
- [SELF: Learning to Filter Noisy Labels with Self-Ensembling](https://mlsanity.com/paper/1910.01842/) (2019)
- [Learning to Learn from Noisy Labeled Data](https://mlsanity.com/paper/1812.05214/) (2018)
- [Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates](https://mlsanity.com/paper/1910.03231/) (2019)
- [Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning](https://mlsanity.com/paper/2006.07805/) (2020)
- [Learning with Symmetric Label Noise: The Importance of Being Unhinged](https://mlsanity.com/paper/1505.07634/) (2015)
- [Iterative Learning with Open-set Noisy Labels](https://mlsanity.com/paper/1804.00092/) (2018)
- [Learning with Biased Complementary Labels](https://mlsanity.com/paper/1711.09535/) (2017)
- [Masking: A New Perspective of Noisy Supervision](https://mlsanity.com/paper/1805.08193/) (2018)
- [Learning from Complementary Labels](https://mlsanity.com/paper/1705.07541/) (2017)

## Built on by (most cited first, 30 of 62)

- [f-Divergence Minimization for Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/2307.15190/) (2023)
- [Robust Training under Label Noise by Over-parameterization](https://mlsanity.com/paper/2202.14026/) (2022)
- [Sample Selection with Uncertainty of Losses for Learning with Noisy Labels](https://mlsanity.com/paper/2106.00445/) (2021)
- [Identifiability of Label Noise Transition Matrix](https://mlsanity.com/paper/2202.02016/) (2022)
- [Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks](https://mlsanity.com/paper/2309.17002/) (2023)
- [Few-shot Learning with Noisy Labels](https://mlsanity.com/paper/2204.05494/) (2022)
- [Understanding Instance-Level Label Noise: Disparate Impacts and Treatments](https://mlsanity.com/paper/2102.05336/) (2021)
- [From Noisy Prediction to True Label: Noisy Prediction Calibration via Generative Model](https://mlsanity.com/paper/2205.00690/) (2022)
- [Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection](https://mlsanity.com/paper/2111.14932/) (2021)
- [Towards Robust Adaptive Object Detection under Noisy Annotations](https://mlsanity.com/paper/2204.02620/) (2022)
- [SimT: Handling Open-set Noise for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2203.15202/) (2022)
- [Instance-Dependent Noisy Label Learning via Graphical Modelling](https://mlsanity.com/paper/2209.00906/) (2022)
- [Label-Noise Robust Diffusion Models](https://mlsanity.com/paper/2402.17517/) (2024)
- [Fine-Grained Classification with Noisy Labels](https://mlsanity.com/paper/2303.02404/) (2023)
- [Being Properly Improper](https://mlsanity.com/paper/2106.09920/) (2021)
- [Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity](https://mlsanity.com/paper/2112.14869/) (2021)
- [Dissecting Sample Hardness: A Fine-Grained Analysis of Hardness Characterization Methods for Data-Centric AI](https://mlsanity.com/paper/2403.04551/) (2024)
- [Hide and Seek in Noise Labels: Noise-Robust Collaborative Active Learning with LLM-Powered Assistance](https://mlsanity.com/paper/2504.02901/) (2025)
- [DyGen: Learning from Noisy Labels via Dynamics-Enhanced Generative Modeling](https://mlsanity.com/paper/2305.19395/) (2023)
- [Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations](https://mlsanity.com/paper/2305.12715/) (2023)
- [Slight Corruption in Pre-training Data Makes Better Diffusion Models](https://mlsanity.com/paper/2405.20494/) (2024)
- [Regroup Median Loss for Combating Label Noise](https://mlsanity.com/paper/2312.06273/) (2023)
- [Impact of Noisy Supervision in Foundation Model Learning](https://mlsanity.com/paper/2403.06869/) (2024)
- [Optical Remote Sensing Image Understanding with Weak Supervision: Concepts, Methods, and Perspectives](https://mlsanity.com/paper/2204.09120/) (2022)
- [Label Noise-Robust Learning using a Confidence-Based Sieving Strategy](https://mlsanity.com/paper/2210.05330/) (2022)
- [ProSelfLC: Progressive Self Label Correction Towards A Low-Temperature Entropy State](https://mlsanity.com/paper/2207.00118/) (2022)
- [De-Confusing Pseudo-Labels in Source-Free Domain Adaptation](https://mlsanity.com/paper/2401.01650/) (2024)
- [Learning under Temporal Label Noise](https://mlsanity.com/paper/2402.04398/) (2024)
- [Conformal Prediction of Classifiers with Many Classes based on Noisy Labels](https://mlsanity.com/paper/2501.12749/) (2025)
- [SelectMix: Enhancing Label Noise Robustness through Targeted Sample Mixing](https://mlsanity.com/paper/2509.11265/) (2025)

## Abstract (arXiv)

Many weakly supervised classification methods employ a noise transition matrix to capture the class-conditional label corruption. To estimate the transition matrix from noisy data, existing methods often need to estimate the noisy class-posterior, which could be unreliable due to the overconfidence of neural networks. In this work, we propose a theoretically grounded method that can estimate the noise transition matrix and learn a classifier simultaneously, without relying on the error-prone noisy class-posterior estimation. Concretely, inspired by the characteristics of the stochastic label corruption process, we propose total variation regularization, which encourages the predicted probabilities to be more distinguishable from each other. Under mild assumptions, the proposed method yields a consistent estimator of the transition matrix. We show the effectiveness of the proposed method through experiments on benchmark and real-world datasets.