mlsanityData to 14 Oct 2026

Robust Temporal Ensembling for Learning with Noisy Labels

arXiv 2109.14563 ↗First posted 29 Sep 2021cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Successful training of deep neural networks with noisy labels is an essential capability as most real-world datasets contain some amount of mislabeled data. Left unmitigated, label noise can sharply degrade typical supervised learning approaches. In this paper, we present robust temporal ensembling (RTE), which combines robust loss with semi-supervised regularization methods to achieve noise-robust learning. We demonstrate that RTE achieves state-of-the-art performance across the CIFAR-10, CIFAR-100, ImageNet, WebVision, and Food-101N datasets, while forgoing the recent trend of label filtering and/or fixing. Finally, we show that RTE also retains competitive corruption robustness to unforeseen input noise using CIFAR-10-C, obtaining a mean corruption error (mCE) of 13.50% even in the presence of an 80% noise ratio, versus 26.9% mCE with standard methods on clean data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 68 references that are in the map. First posted on arXiv 29 Sep 2021.

Builds on

44 of 68 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2016Wide Residual Networkscited by 3,070
  5. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  6. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  7. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  8. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  9. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  10. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  11. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  12. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
32 more
  1. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  2. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  3. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  4. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
  5. 2018Learning to Reweight Examples for Robust Deep Learningcited by 675
  6. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  7. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
  8. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 605
  9. 2017Deep Learning Scaling is Predictable, Empiricallycited by 582
  10. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  11. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  12. 2017Population Based Training of Neural Networkscited by 367
  13. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 328
  14. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  15. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  16. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  17. 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
  18. 2019Understanding and Utilizing Deep Neural Networks Trained with Noisy Labelscited by 209
  19. 2017CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noisecited by 181
  20. 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
  21. 2017Learning from Noisy Labels with Distillationcited by 161
  22. 2018Learning to Learn from Noisy Labeled Datacited by 160
  23. 2016Improving the Robustness of Deep Neural Networks via Stability Trainingcited by 156
  24. 2019Population Based Augmentation: Efficient Learning of Augmentation Policy Schedulescited by 154
  25. 2019Deep Self-Learning From Noisy Labelscited by 122
  26. 2018Iterative Learning with Open-set Noisy Labelscited by 108
  27. 2019Combating Label Noise in Deep Learning Using Abstentioncited by 86
  28. 2018There Are Many Consistent Explanations of Unlabeled Data: Why You Should Averagecited by 82
  29. 2019Robust Inference via Generative Classifiers for Handling Noisy Labelscited by 68
  30. 2018Deep Bilevel Learningcited by 49
  31. 2019A Generalized Framework for Population Based Trainingcited by 22
  32. 2014Google matrix analysis of directed networkscited by 2

Built on by

Not cited yet.