Tackling Instance-Dependent Label Noise with Dynamic Distribution Calibration
Not on the map. Why?
Abstract (arXiv)
Instance-dependent label noise is realistic but rather challenging, where the label-corruption process depends on instances directly. It causes a severe distribution shift between the distributions of training and test data, which impairs the generalization of trained models. Prior works put great effort into tackling the issue. Unfortunately, these works always highly rely on strong assumptions or remain heuristic without theoretical guarantees. In this paper, to address the distribution shift in learning with instance-dependent label noise, a dynamic distribution-calibration strategy is adopted. Specifically, we hypothesize that, before training data are corrupted by label noise, each class conforms to a multivariate Gaussian distribution at the feature level. Label noise produces outliers to shift the Gaussian distribution. During training, to calibrate the shifted distribution, we propose two methods based on the mean and covariance of multivariate Gaussian distribution respectively. The mean-based method works in a recursive dimension-reduction manner for robust mean estimation, which is theoretically guaranteed to train a high-quality model against label noise. The covariance-based method works in a distribution disturbance manner, which is experimentally verified to improve the model robustness. We demonstrate the utility and effectiveness of our methods on datasets with synthetic label noise and real-world unknown noise.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 59 of its 76 references that are in the map. First posted on arXiv 11 Oct 2022.
Builds on
59 of 76 references.
References · 59
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2017Squeeze-and-Excitation Networkscited by 4,789
- 2013Auto-Encoding Variational Bayescited by 4,053
- 2016Wide Residual Networkscited by 3,070
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
- 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
- 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
- 2017A Closer Look at Memorization in Deep Networkscited by 942
- 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
- 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
- 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
- 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 500
38 more
- 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 416
- 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 413
- 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
- 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
- 2020Identifying Mislabeled Data using the Area Under the Margin Rankingcited by 261
- 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
- 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
- 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
- 2019Understanding and Utilizing Deep Neural Networks Trained with Noisy Labelscited by 209
- 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
- 2020Does label smoothing mitigate label noise?cited by 204
- 2017CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noisecited by 181
- 2021Free Lunch for Few-shot Learning: Distribution Calibrationcited by 165
- 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
- 2017Learning from Noisy Labels with Distillationcited by 161
- 2020Learning with Instance-Dependent Label Noise: A Sample Sieve Approachcited by 158
- 2019NLNL: Negative Learning for Noisy Labelscited by 151
- 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
- 2016Agnostic Estimation of Mean and Covariancecited by 131
- 2022Uncertainty Modeling for Out-of-Distribution Generalizationcited by 125
- 2019Implicit Semantic Data Augmentation for Deep Networkscited by 106
- 2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
- 2019Recent Advances in Algorithmic High-Dimensional Robust Statisticscited by 101
- 2021Provably End-to-end Label-Noise Learning without Anchor Pointscited by 97
- 2018Masking: A New Perspective of Noisy Supervisioncited by 93
- 2020A Second-Order Approach to Learning with Instance-Dependent Label Noisecited by 90
- 2019Curriculum Loss: Robust Learning and Generalization against Label Corruptioncited by 88
- 2020Multi-Objective Interpolation Training for Robustness to Label Noisecited by 86
- 2021Augmentation Strategies for Learning with Noisy Labelscited by 86
- 2020Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noisecited by 82
- 2021FINE Samples for Learning with Noisy Labelscited by 82
- 2021Jo-SRC: A Contrastive Approach for Combating Noisy Labelscited by 80
- 2021Sample Selection with Uncertainty of Losses for Learning with Noisy Labelscited by 79
- 2020Error-Bounded Correction of Noisy Labelscited by 74
- 2019Robust Inference via Generative Classifiers for Handling Noisy Labelscited by 68
- 2020Confidence Scores Make Instance-dependent Label-noise Learning Possiblecited by 63
- 2021Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularizationcited by 62
- 2021NGC: A Unified Framework for Learning with Open-World Noisy Datacited by 58
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 1.
2 papers.