Approximating Instance-Dependent Noise via Instance-Confidence Embedding
Not on the map. Why?
Abstract (arXiv)
Label noise in multiclass classification is a major obstacle to the deployment of learning systems. However, unlike the widely used class-conditional noise (CCN) assumption that the noisy label is independent of the input feature given the true label, label noise in real-world datasets can be aleatory and heavily dependent on individual instances. In this work, we investigate the instance-dependent noise (IDN) model and propose an efficient approximation of IDN to capture the instance-specific label corruption. Concretely, noting the fact that most columns of the IDN transition matrix have only limited influence on the class-posterior estimation, we propose a variational approximation that uses a single-scalar confidence parameter. To cope with the situation where the mapping from the instance to its confidence value could vary significantly for two adjacent instances, we suggest using instance embedding that assigns a trainable parameter to each instance. The resulting instance-confidence embedding (ICE) method not only performs well under label noise but also can effectively detect ambiguous or mislabeled instances. We validate its utility on various image and text classification tasks.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 52 of its 76 references that are in the map. First posted on arXiv 25 Mar 2021.
Builds on
52 of 76 references.
References · 52
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
38 more
- 2018Deep contextualized word representationscited by 2,285
- 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
- 2016Understanding deep learning requires rethinking generalizationcited by 1,657
- 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
- 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
- 2018Neural Network Acceptability Judgmentscited by 952
- 2017A Closer Look at Memorization in Deep Networkscited by 936
- 2016Deep Variational Information Bottleneckcited by 915
- 2015Deep Learning and the Information Bottleneck Principlecited by 793
- 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
- 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
- 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
- 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
- 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
- 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
- 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
- 2018Deep Learning for Classical Japanese Literaturecited by 415
- 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
- 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 411
- 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
- 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
- 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
- 2020Consistent Estimators for Learning to Defer to an Expertcited by 262
- 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
- 2017Decoupling "when to update" from "how to update"cited by 230
- 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
- 2020Does label smoothing mitigate label noise?cited by 204
- 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
- 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
- 2020Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learningcited by 142
- 2015Learning with Symmetric Label Noise: The Importance of Being Unhingedcited by 120
- 2018Iterative Learning with Open-set Noisy Labelscited by 108
- 2019Curriculum Loss: Robust Learning and Generalization against Label Corruptioncited by 87
- 2019Combating Label Noise in Deep Learning Using Abstentioncited by 86
- 2020Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noisecited by 82
- 2020A Topological Filter for Learning with Label Noisecited by 76
- 2019Data Cleansing for Models Trained with SGDcited by 67
- 2020Intra Order-preserving Functions for Calibration of Multi-Class Neural Networkscited by 65
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2021: 1; 2022: 0; 2023: 0; 2024: 0; 2025: 2.
3 papers.