Do We Need to Penalize Variance of Losses for Learning with Label Noise?
Not on the map. Why?
Abstract (arXiv)
Algorithms which minimize the averaged loss have been widely designed for dealing with noisy labels. Intuitively, when there is a finite training sample, penalizing the variance of losses will improve the stability and generalization of the algorithms. Interestingly, we found that the variance should be increased for the problem of learning with noisy labels. Specifically, increasing the variance will boost the memorization effects and reduce the harmfulness of incorrect labels. By exploiting the label noise transition matrix, regularizers can be easily designed to reduce the variance of losses and be plugged in many existing algorithms. Empirically, the proposed method by increasing the variance of losses significantly improves the generalization ability of baselines on both synthetic and real-world datasets.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 49 of its 96 references that are in the map. First posted on arXiv 30 Jan 2022.
Builds on
49 of 96 references.
References · 49
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
- 2016Understanding deep learning requires rethinking generalizationcited by 1,657
- 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
- 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
- 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
- 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
- 2017A Closer Look at Memorization in Deep Networkscited by 936
- 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
- 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
- 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
- 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
37 more
- 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
- 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
- 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
- 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
- 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
- 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
- 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
- 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
- 2019Causality for Machine Learningcited by 242
- 2017Decoupling "when to update" from "how to update"cited by 230
- 2019Understanding and Utilizing Deep Neural Networks Trained with Noisy Labelscited by 209
- 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
- 2018Dimensionality-Driven Learning with Noisy Labelscited by 173
- 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
- 2017Learning from Noisy Labels with Distillationcited by 160
- 2019Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networkscited by 158
- 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
- 2020Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learningcited by 142
- 2018CurriculumNet: Weakly Supervised Learning from Large-Scale Web Imagescited by 126
- 2015Learning with Symmetric Label Noise: The Importance of Being Unhingedcited by 120
- 2017Learning From Noisy Large-Scale Datasets With Minimal Supervisioncited by 115
- 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 112
- 2016Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learningcited by 106
- 2021Provably End-to-end Label-Noise Learning without Anchor Pointscited by 97
- 2018Masking: A New Perspective of Noisy Supervisioncited by 93
- 2017Toward Robustness against Label Noise in Training Deep Discriminative Neural Networkscited by 78
- 2016Variance Reduction for Faster Non-Convex Optimizationcited by 72
- 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
- 2017Learning with Bounded Instance- and Label-dependent Label Noisecited by 57
- 2014Making Risk Minimization Tolerant to Label Noisecited by 54
- 2017Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labelscited by 52
- 2021Instance-dependent Label-noise Learning under a Structural Causal Modelcited by 50
- 2021Robust Long-Tailed Learning under Label Noisecited by 35
- 2021Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Networkcited by 30
- 2018Robustness of Conditional GANs to Noisy Labelscited by 27
- 2020Learning Adaptive Loss for Robust Learning with Noisy Labelscited by 10
- 2015SGD with Variance Reduction beyond Empirical Risk Minimizationcited by 2
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.