mlsanityData to 7 Oct 2026

Identifying Mislabeled Data using the Area Under the Margin Ranking

arXiv 2001.10528 ↗First posted 28 Jan 2020cs.LG cs.CV stat.MLcited by 259

Not on the map. Why?

Abstract (arXiv)

Not all data in a typical training set help with generalization; some samples can be overly ambiguous or outrightly mislabeled. This paper introduces a new method to identify such samples and mitigate their impact when training neural networks. At the heart of our algorithm is the Area Under the Margin (AUM) statistic, which exploits differences in the training dynamics of clean and mislabeled samples. A simple procedure - adding an extra class populated with purposefully mislabeled threshold samples - learns a AUM upper bound that isolates mislabeled data. This approach consistently improves upon prior work on synthetic and real-world datasets. On the WebVision50 classification task our method removes 17% of training data, yielding a 1.6% (absolute) improvement in test error. On CIFAR100 removing 13% of the data leads to a 1.2% drop in error.

Key facts. Cited by 259 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 31; 2022: 46; 2023: 43; 2024: 45; 2025: 63; 2026: 31.

259 papers.

Sort:

By area:

  1. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
  2. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 288
  3. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 233
  4. 2022Training Data Influence Analysis and Estimation: A Surveycited by 134
  5. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  6. 2022Coverage-centric Coreset Selection for High Pruning Ratescited by 115
  7. 2021Deep Learning with Label Differential Privacycited by 113
  8. 2021FINE Samples for Learning with Noisy Labelscited by 82
  9. 2021Sample Selection with Uncertainty of Losses for Learning with Noisy Labelscited by 79
  10. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  11. 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
  12. 2021Learning Fast Sample Re-weighting Without Reward Datacited by 69
38 more
  1. 2023D2 Pruning: Message Passing for Balancing Diversity and Difficulty in Data Pruningcited by 68
  2. 2021NGC: A Unified Framework for Learning with Open-World Noisy Datacited by 58
  3. 2022On Uncertainty, Tempering, and Data Augmentation in Bayesian Classificationcited by 46
  4. 2022Learning to Detect Mobile Objects from LiDAR Scans Without Labelscited by 44
  5. 2021LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environmentcited by 40
  6. 2021Faster Meta Update Strategy for Noise-Robust Deep Learningcited by 38
  7. 2023Spanning Training Progress: Temporal Dual-Depth Scoring (TDDS) for Enhanced Dataset Pruningcited by 38
  8. 2022Characterizing Datapoints via Second-Split Forgettingcited by 37
  9. 2023Self-Influence Guided Data Reweighting for Language Model Pre-trainingcited by 37
  10. 2021Robust Long-Tailed Learning under Label Noisecited by 35
  11. 2023Data Valuation Without Training of a Modelcited by 35
  12. 2022On the Calibration of Pre-trained Language Models using Mixup Guided by Area Under the Margin and Saliencycited by 31
  13. 2022A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainabilitycited by 30
  14. 2022Robust Training of Graph Neural Networks via Noise Governancecited by 29
  15. 2025A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advancescited by 28
  16. 2021Continual Learning on Noisy Data Streams via Self-Purified Replaycited by 27
  17. 2022Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamicscited by 26
  18. 2024Corrective Machine Unlearningcited by 26
  19. 2023Memorization Through the Lens of Curvature of Loss Function Around Samplescited by 24
  20. 2023Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidancecited by 24
  21. 2024Concept-skill Transferability-based Data Selection for Large Vision-Language Modelscited by 24
  22. 2022Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular datacited by 22
  23. 2025Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertaintycited by 22
  24. 2023Learning from Noisy Labels with Decoupled Meta Label Purifiercited by 21
  25. 2021Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrectioncited by 20
  26. 2023Neural Relation Graph: A Unified Framework for Identifying Label Noise and Outlier Datacited by 20
  27. 2023You Only Condense Once: Two Rules for Pruning Condensed Datasetscited by 20
  28. 2024Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performancecited by 20
  29. 2021A Tale Of Two Long Tailscited by 19
  30. 2022Deconstructing Distributions: A Pointwise Framework of Learningcited by 19
  31. 2024Are Large-scale Soft Labels Necessary for Large-scale Dataset Distillation?cited by 19
  32. 2023Make Every Example Count: On the Stability and Utility of Self-Influence for Learning from Noisy NLP Datasetscited by 18
  33. 2021PropMix: Hard Sample Filtering and Proportional MixUp for Learning with Noisy Labelscited by 17
  34. 2021MOTIF: A Large Malware Reference Dataset with Ground Truth Family Labelscited by 17
  35. 2022Learning Multimodal Data Augmentation in Feature Spacecited by 17
  36. 2023Late Stopping: Avoiding Confidently Learning from Mislabeled Examplescited by 17
  37. 2022Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Treescited by 16
  38. 2023AQuA: A Benchmarking Tool for Label Quality Assessmentcited by 16

50 most cited of 259 All 259 (JSON)