mlsanityData to 14 Oct 2026

Regularly Truncated M-estimators for Learning with Noisy Labels

arXiv 2309.00894 ↗First posted 2 Sep 2023cs.LG cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

The sample selection approach is very popular in learning with noisy labels. As deep networks learn pattern first, prior methods built on sample selection share a similar training procedure: the small-loss examples can be regarded as clean examples and used for helping generalization, while the large-loss examples are treated as mislabeled ones and excluded from network parameter updates. However, such a procedure is arguably debatable from two folds: (a) it does not consider the bad influence of noisy labels in selected small-loss examples; (b) it does not make good use of the discarded large-loss examples, which may be clean or have meaningful information for generalization. In this paper, we propose regularly truncated M-estimators (RTME) to address the above two issues simultaneously. Specifically, RTME can alternately switch modes between truncated M-estimators and original M-estimators. The former can adaptively select small-losses examples without knowing the noise rate and reduce the side-effects of noisy labels in them. The latter makes the possibly clean examples but with large losses involved to help generalization. Theoretically, we demonstrate that our strategies are label-noise-tolerant. Empirically, comprehensive experimental results show that our method can outperform multiple baselines and is robust to broad noise types and levels.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 61 of its 102 references that are in the map. First posted on arXiv 2 Sep 2023.

Builds on

61 of 102 references.

References · 61

  1. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  2. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  3. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  4. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  5. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  6. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  7. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  8. 2017A Closer Look at Memorization in Deep Networkscited by 942
  9. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
  10. 2018Learning to Reweight Examples for Robust Deep Learningcited by 675
  11. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  12. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
38 more
  1. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  2. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 500
  3. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 443
  4. 2023Trained Transformers Learn Linear Models In-Contextcited by 337
  5. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 328
  6. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  7. 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
  8. 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
  9. 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
  10. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 256
  11. 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
  12. 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
  13. 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
  14. 2017Learning from Noisy Labels with Distillationcited by 161
  15. 2019NLNL: Negative Learning for Noisy Labelscited by 151
  16. 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
  17. 2020Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learningcited by 142
  18. 2015A Light CNN for Deep Face Representation with Noisy Labelscited by 126
  19. 2020A Survey of Label-noise Representation Learning: Past, Present and Futurecited by 122
  20. 2022Selective-Supervised Contrastive Learning with Noisy Labelscited by 112
  21. 2018Iterative Learning with Open-set Noisy Labelscited by 108
  22. 2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
  23. 2021Provably End-to-end Label-Noise Learning without Anchor Pointscited by 97
  24. 2020A Second-Order Approach to Learning with Instance-Dependent Label Noisecited by 90
  25. 2019Curriculum Loss: Robust Learning and Generalization against Label Corruptioncited by 88
  26. 2021Sample Selection with Uncertainty of Losses for Learning with Noisy Labelscited by 79
  27. 2017Toward Robustness against Label Noise in Training Deep Discriminative Neural Networkscited by 78
  28. 2019Robust Inference via Generative Classifiers for Handling Noisy Labelscited by 68
  29. 2021Clusterability as an Alternative to Anchor Points When Learning with Noisy Labelscited by 65
  30. 2022Towards Robust Graph Neural Networks for Noisy Graphs with Sparse Labelscited by 63
  31. 2017Learning with Bounded Instance- and Label-dependent Label Noisecited by 57
  32. 2022Learning with Neighbor Consistency for Noisy Labelscited by 55
  33. 2019On Symmetric Losses for Learning from Corrupted Labelscited by 53
  34. 2022ProMix: Combating Label Noise via Maximizing Clean Sample Utilitycited by 48
  35. 2022Mitigating Memorization of Noisy Labels by Clipping the Model Predictioncited by 40
  36. 2021Robust Long-Tailed Learning under Label Noisecited by 36
  37. 2022Identifiability of Label Noise Transition Matrixcited by 34
  38. 2021Exploring Set Similarity for Dense Self-supervised Representation Learningcited by 29

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 4; 2026: 1.

8 papers.

  1. 2025Early Stopping Against Label Noise Without Validation Datacited by 21
  2. 2023ERASE: Error-Resilient Representation Learning on Graphs for Label Noise Tolerancecited by 11
  3. 2024Meta-Learning Guided Label Noise Distillation for Robust Signal Modulation Classificationcited by 4
  4. 2025Enhancing Sample Selection Against Label Noise by Cutting Mislabeled Easy Examplescited by 4
  5. 2025On the Role of Label Noise in the Feature Learning Processcited by 4
  6. 2025Multi-Modal Dataset Distillation in the Wildcited by 1
  7. 2024EchoAlign: Bridging Generative and Discriminative Learning under Noisy Labelscited by 0
  8. 2026FedCova: Robust Federated Covariance Learning Against Noisy Labelscited by 0