mlsanityData to 7 Oct 2026

An Instance-Dependent Simulation Framework for Learning with Label Noise

arXiv 2107.11413 ↗First posted 23 Jul 2021cs.LG cs.HCcited by 14

Not on the map. Why?

Abstract (arXiv)

We propose a simulation framework for generating instance-dependent noisy labels via a pseudo-labeling paradigm. We show that the distribution of the synthetic noisy labels generated with our framework is closer to human labels compared to independent and class-conditional random flipping. Equipped with controllable label noise, we study the negative impact of noisy labels across a few practical settings to understand when label noise is more problematic. We also benchmark several existing algorithms for learning with noisy labels and compare their behavior on our synthetic datasets and on the datasets with independent random label noise. Additionally, with the availability of annotator information from our simulation framework, we propose a new technique, Label Quality Model (LQM), that leverages annotator features to predict and correct against noisy labels. We show that by adding LQM as a label correction step before applying existing noisy label techniques, we can further improve the models' performance.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 52 of its 81 references that are in the map. First posted on arXiv 23 Jul 2021.

Builds on

52 of 81 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2014Going Deeper with Convolutionscited by 4,231
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  9. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  10. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  11. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  12. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
38 more
  1. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  2. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  3. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  4. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  5. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
  6. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  7. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  8. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  9. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
  10. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  11. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  12. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 411
  13. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  14. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 388
  15. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  16. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
  17. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  18. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  19. 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
  20. 2017High-dimensional dynamics of generalization error in neural networkscited by 238
  21. 2019Human uncertainty makes classification more robustcited by 231
  22. 2017Decoupling "when to update" from "how to update"cited by 230
  23. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  24. 2017CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noisecited by 181
  25. 2017Deep Learning is Robust to Massive Label Noisecited by 164
  26. 2017Learning from Noisy Labels with Distillationcited by 160
  27. 2019Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networkscited by 158
  28. 2017Learning From Noisy Large-Scale Datasets With Minimal Supervisioncited by 115
  29. 2018Iterative Learning with Open-set Noisy Labelscited by 108
  30. 2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
  31. 2020A Second-Order Approach to Learning with Instance-Dependent Label Noisecited by 90
  32. 2020Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noisecited by 82
  33. 2019Robust Inference via Generative Classifiers for Handling Noisy Labelscited by 67
  34. 2014Spectral Methods meet EM: A Provably Optimal Algorithm for Crowdsourcingcited by 63
  35. 2020Confidence Scores Make Instance-dependent Label-noise Learning Possiblecited by 63
  36. 2017Learning From Noisy Singly-labeled Datacited by 50
  37. 2015The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognitioncited by 48
  38. 2020Strength from Weakness: Fast Learning Using Weak Supervisioncited by 18

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 5; 2024: 3; 2025: 0; 2026: 1.

14 papers.

Sort:

  1. 2021A Fine-Grained Analysis on Distribution Shiftcited by 171
  2. 2022Detecting Label Errors in Token Classification Datacited by 12
  3. 2023Multi-annotator Deep Learning: A Probabilistic Framework for Classificationcited by 8
  4. 2022Meta Self-Refinement for Robust Learning with Weak Supervisioncited by 7
  5. 2022Bootstrapping the Relationship Between Images and Their Clean and Noisy Labelscited by 7
  6. 2023Making Binary Classification from Multiple Unlabeled Datasets Almost Free of Supervisioncited by 4
  7. 2022Quantifying the Impact of Label Noise on Federated Learningcited by 3
  8. 2023Label Smarter, Not Harder: CleverLabel for Faster Annotation of Ambiguous Image Classification with Higher Qualitycited by 3
  9. 2024Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extensioncited by 3
  10. 2023Generating the Ground Truth: Synthetic Data for Soft Label and Label Noise Researchcited by 2
  11. 2024AlleNoise: large-scale text classification benchmark dataset with real-world label noisecited by 2
  12. 2023Price of Stability in Quality-Aware Federated Learningcited by 0
2 more
  1. 2024EchoAlign: Bridging Generative and Discriminative Learning under Noisy Labelscited by 0
  2. 2026Benchmarking Instance-Dependent Label Noise with Controlled Corruptionscited by 0