Logistic-Normal Likelihoods for Heteroscedastic Label Noise
Not on the map. Why?
Abstract (arXiv)
A natural way of estimating heteroscedastic label noise in regression is to model the observed (potentially noisy) target as a sample from a normal distribution, whose parameters can be learned by minimizing the negative log-likelihood. This formulation has desirable loss attenuation properties, as it reduces the contribution of high-error examples. Intuitively, this behavior can improve robustness against label noise by reducing overfitting. We propose an extension of this simple and probabilistic approach to classification that has the same desirable loss attenuation properties. Furthermore, we discuss and address some practical challenges of this extension. We evaluate the effectiveness of the method by measuring its robustness against label noise in classification. We perform enlightening experiments exploring the inner workings of the method, including sensitivity to hyperparameters, ablation studies, and other insightful analyses.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 6 Apr 2023.
Builds on
30 of 46 references.
References · 30
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
- 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
- 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
- 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
- 2017A Closer Look at Memorization in Deep Networkscited by 942
- 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
- 2018Learning to Reweight Examples for Robust Deep Learningcited by 675
- 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
- 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
- 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 500
18 more
- 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 443
- 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 416
- 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
- 2020Are we done with ImageNet?cited by 333
- 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
- 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 256
- 2020Does label smoothing mitigate label noise?cited by 204
- 2019Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networkscited by 159
- 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 113
- 2017TensorFlow Distributionscited by 106
- 2019Combating Label Noise in Deep Learning Using Abstentioncited by 86
- 2021Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labelscited by 74
- 2019Robust Bi-Tempered Logistic Loss Based on Bregman Divergencescited by 61
- 2021Asymmetric Loss Functions for Learning with Noisy Labelscited by 46
- 2020When Optimizing f-divergence is Robust with Label Noisecited by 44
- 2021Correlated Input-Dependent Label Noise in Large-Scale Image Classificationcited by 37
- 2021On the Robustness of Monte Carlo Dropout Trained with Noisy Labelscited by 11
- 2021Deep Classifiers with Label Noise Modeling and Distance Awarenesscited by 8
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.