mlsanityData to 14 Oct 2026

Deep Learning is Robust to Massive Label Noise

arXiv 1705.10694 ↗First posted 30 May 2017cs.LG cs.AI cs.CVcited by 165

Not on the map. Why?

Abstract (arXiv)

Deep neural networks trained on large supervised datasets have led to impressive results in image classification and other tasks. However, well-annotated datasets can be time-consuming and expensive to collect, lending increased interest to larger but noisy datasets that are more easily obtained. In this paper, we show that deep neural networks are capable of generalizing from training data for which true labels are massively outnumbered by incorrect labels. We demonstrate remarkably high test performance after training on corrupted data from MNIST, CIFAR, and ImageNet. For example, on MNIST we obtain test accuracy above 90 percent even after each clean training example has been diluted with 100 randomly-labeled examples. Such behavior holds across multiple patterns of label noise, even when erroneous labels are biased towards confusing classes. We show that training in this regime requires a significant but manageable increase in dataset size that is related to the factor by which correct labels have been diluted. Finally, we provide an analysis of our results that shows how increasing noise decreases the effective batch size.

Key facts. Cited by 165 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 40; 2022: 28; 2023: 28; 2024: 26; 2025: 23; 2026: 20.

165 papers.

Sort:

By area:

  1. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 395
  2. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  3. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
  4. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 236
  5. 2021MIST: Multiple Instance Self-Training Framework for Video Anomaly Detectioncited by 138
  6. 2022Coverage-centric Coreset Selection for High Pruning Ratescited by 117
  7. 2022How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervisioncited by 102
  8. 2025Maximizing Confidence Alone Improves Reasoningcited by 100
  9. 2021Multi-Label Learning from Single Positive Labelscited by 81
  10. 2021On the geometry of generalization and memorization in deep neural networkscited by 73
  11. 2021Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advancescited by 54
  12. 2022MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Promptingcited by 41
38 more
  1. 2021HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networkscited by 38
  2. 2021Towards Understanding Deep Learning from Noisy Labels with Small-Loss Criterioncited by 36
  3. 2024Cross-Entropy Is All You Need To Invert the Data Generating Processcited by 31
  4. 2025Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challengescited by 31
  5. 2021Mitigating Memorization of Noisy Labels via Regularization between Representationscited by 26
  6. 2022On the Generalization Mystery in Deep Learningcited by 25
  7. 2022Indiscriminate Data Poisoning Attacks on Neural Networkscited by 25
  8. 2021Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonizationcited by 24
  9. 2021DeepCervix: A Deep Learning-based Framework for the Classification of Cervical Cells Using Hybrid Deep Feature Fusion Techniquescited by 23
  10. 2024Why is SAM Robust to Label Noise?cited by 21
  11. 2024Towards Data-Centric RLHF: Simple Metrics for Preference Dataset Comparisoncited by 21
  12. 2021RATT: Leveraging Unlabeled Data to Guarantee Generalizationcited by 20
  13. 2022FUN-SIS: a Fully UNsupervised approach for Surgical Instrument Segmentationcited by 20
  14. 2022Identifying Hard Noise in Long-Tailed Sample Distributioncited by 19
  15. 2023Make Every Example Count: On the Stability and Utility of Self-Influence for Learning from Noisy NLP Datasetscited by 19
  16. 20233D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capturecited by 18
  17. 2021Self-Adaptive Training: Bridging Supervised and Self-Supervised Learningcited by 17
  18. 2022CTRL: Clustering Training Losses for Label Error Detectioncited by 17
  19. 2023The Dataset Multiplicity Problem: How Unreliable Data Impacts Predictionscited by 17
  20. 2023Late Stopping: Avoiding Confidently Learning from Mislabeled Examplescited by 17
  21. 2021Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyondcited by 16
  22. 2022Detecting Label Errors by using Pre-Trained Language Modelscited by 16
  23. 2021Saliency Grafting: Innocuous Attribution-Guided Mixup with Calibrated Label Mixingcited by 15
  24. 2022Robustness to Label Noise Depends on the Shape of the Noise Distribution in Feature Spacecited by 15
  25. 2021An Instance-Dependent Simulation Framework for Learning with Label Noisecited by 14
  26. 2021ScanMix: Learning from Severe Label Noise via Semantic Clustering and Semi-Supervised Learningcited by 13
  27. 2022Non-Markovian Reward Modelling from Trajectory Labels via Interpretable Multiple Instance Learningcited by 13
  28. 2024The METRIC-framework for assessing data quality for trustworthy AI in medicine: a systematic reviewcited by 13
  29. 2024A noisy elephant in the room: Is your out-of-distribution detector robust to label noise?cited by 13
  30. 2021Benchmarking of eight recurrent neural network variants for breath phase and adventitious sound detection on a self-developed open-access lung sound database-HF_Lung_V1cited by 12
  31. 2021A Privacy-Preserving Approach to Extraction of Personal Information through Automatic Annotation and Federated Learningcited by 12
  32. 2022The Curious Case of Benign Memorizationcited by 12
  33. 2023Semantic Scene Completion with Cleaner Selfcited by 12
  34. 2021Estimating the electrical power output of industrial devices with end-to-end time-series classification in the presence of label noisecited by 10
  35. 2022Generalization Through The Lens Of Leave-One-Out Errorcited by 10
  36. 2023Unsupervised classification to improve the quality of a bird song recording datasetcited by 10
  37. 2023When does Privileged Information Explain Away Label Noise?cited by 10
  38. 2021Weakly Supervised Change Detection Using Guided Anisotropic Difusioncited by 8

50 most cited of 165 All 165 (JSON)