mlsanityData to 7 Oct 2026

Searching for Robustness: Loss Learning for Noisy Classification Tasks

arXiv 2103.00243 ↗First posted 27 Feb 2021cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

We present a "learning to learn" approach for automatically constructing white-box classification loss functions that are robust to label noise in the training data. We parameterize a flexible family of loss functions using Taylor polynomials, and apply evolutionary strategies to search for noise-robust losses in this space. To learn re-usable loss functions that can apply to new tasks, our fitness function scores their performance in aggregate across a range of training dataset and architecture combinations. The resulting white-box loss provides a simple and fast "plug-and-play" module that enables effective noise-robust learning in diverse downstream tasks, without requiring a special training procedure or network architecture. The efficacy of our method is demonstrated on a variety of datasets with both synthetic and real label noise, where we compare favourably to previous work.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 39 of its 54 references that are in the map. First posted on arXiv 27 Feb 2021.

Builds on

39 of 54 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  7. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  8. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  9. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  10. 2019When Does Label Smoothing Help?cited by 985
  11. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  12. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
27 more
  1. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  2. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  3. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  4. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  5. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 542
  6. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
  7. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
  8. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  9. 2018Deep Learning for Classical Japanese Literaturecited by 415
  10. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
  11. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  12. 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
  13. 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
  14. 2018Learning to Learn from Noisy Labeled Datacited by 160
  15. 2019Feature-Critic Networks for Heterogeneous Domain Generalizationcited by 130
  16. 2017Learned Optimizers that Scale and Generalizecited by 109
  17. 2015Webly Supervised Learning of Convolutional Networkscited by 95
  18. 2018Evolved Policy Gradientscited by 86
  19. 2019Improving Generalization in Meta Reinforcement Learning using Learned Objectivescited by 68
  20. 2019Robust Bi-Tempered Logistic Loss Based on Bregman Divergencescited by 61
  21. 2019Meta-Learning via Learned Losscited by 60
  22. 2018Deep Bilevel Learningcited by 48
  23. 2018Learning to Teach with Dynamic Loss Functionscited by 40
  24. 2019Addressing the Loss-Metric Mismatch with Adaptive Loss Alignmentcited by 33
  25. 2019Improved Training Speed, Accuracy, and Data Utilization Through Loss Function Optimizationcited by 28
  26. 2020Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labelscited by 20
  27. 2017Two-temperature logistic regression based on the Tsallis divergencecited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 3; 2024: 4; 2025: 2; 2026: 1.

13 papers.

Sort:

  1. 2022Learning Symbolic Model-Agnostic Loss Functions via Meta-Learningcited by 9
  2. 2022Meta Mirror Descent: Optimiser Learning for Fast Convergencecited by 7
  3. 2022Truncate-Split-Contrast: A Framework for Learning from Mislabeled Videoscited by 4
  4. 2023Meta-Learning Adaptive Loss Functionscited by 4
  5. 2023DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learningcited by 4
  6. 2023Improve Noise Tolerance of Robust Loss via Noise-Awarenesscited by 3
  7. 2024Neural Loss Function Evolution for Large-Scale Image Classifier Convolutional Neural Networkscited by 2
  8. 2024Fast and Efficient Local Search for Genetic Programming Based Loss Function Learningcited by 1
  9. 2024Data augmentation with automated machine learning: approaches and performance comparison with classical data augmentation methodscited by 1
  10. 2024Evolving Loss Functions for Specific Image Augmentation Techniquescited by 0
  11. 2025Meta-Sparsity: Learning Optimal Sparse Structures in Multi-task Networks through Meta-learningcited by 0
  12. 2025Label-Noise Resistant Learning via Optimal Brain Damage Maskingcited by 0
1 more
  1. 2026Leveraging Data Symmetries to Select an Optimal Subset of Training Data under Label Noisecited by 0