# Curriculum Loss: Robust Learning and Generalization against Label Corruption

arXiv: https://arxiv.org/abs/1905.10045 · HTML: https://mlsanity.com/paper/1905.10045/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2019.

## Built on by (most cited first, 30 of 88)

- [Understanding and Improving Early Stopping for Learning with Noisy Labels](https://mlsanity.com/paper/2106.15853/) (2021)
- [Sample Selection with Uncertainty of Losses for Learning with Noisy Labels](https://mlsanity.com/paper/2106.00445/) (2021)
- [IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing](https://mlsanity.com/paper/2301.13359/) (2023)
- [Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization](https://mlsanity.com/paper/2102.02414/) (2021)
- [Open-set Label Noise Can Improve Robustness Against Inherent Label Noise](https://mlsanity.com/paper/2106.10891/) (2021)
- [Instance-Dependent Label-Noise Learning with Manifold-Regularized Transition Matrix Estimation](https://mlsanity.com/paper/2206.02791/) (2022)
- [Asymmetric Loss Functions for Learning with Noisy Labels](https://mlsanity.com/paper/2106.03110/) (2021)
- [RIME: Robust Preference-based Reinforcement Learning with Noisy Preferences](https://mlsanity.com/paper/2402.17257/) (2024)
- [Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Network](https://mlsanity.com/paper/2105.13001/) (2021)
- [Continual Learning on Noisy Data Streams via Self-Purified Replay](https://mlsanity.com/paper/2110.07735/) (2021)
- [CLNode: Curriculum Learning for Node Classification](https://mlsanity.com/paper/2206.07258/) (2022)
- [Learning with Noisy Labels for Robust Point Cloud Segmentation](https://mlsanity.com/paper/2107.14230/) (2021)
- [Adaptive Sample Selection for Robust Learning under Label Noise](https://mlsanity.com/paper/2106.15292/) (2021)
- [SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels](https://mlsanity.com/paper/2308.12064/) (2023)
- [Learning to Segment from Noisy Annotations: A Spatial Correction Approach](https://mlsanity.com/paper/2308.02498/) (2023)
- [Late Stopping: Avoiding Confidently Learning from Mislabeled Examples](https://mlsanity.com/paper/2308.13862/) (2023)
- [Uncertainty-aware Clustering for Unsupervised Domain Adaptive Object Re-identification](https://mlsanity.com/paper/2108.09682/) (2021)
- [Scalable Penalized Regression for Noise Detection in Learning with Noisy Labels](https://mlsanity.com/paper/2203.07788/) (2022)
- [Improving Medical Image Classification with Label Noise Using Dual-uncertainty Estimation](https://mlsanity.com/paper/2103.00528/) (2021)
- [Model Generalization on COVID-19 Fake News Detection](https://mlsanity.com/paper/2101.03841/) (2021)
- [On the Robustness of Monte Carlo Dropout Trained with Noisy Labels](https://mlsanity.com/paper/2103.12002/) (2021)
- [Evaluating Multi-label Classifiers with Noisy Labels](https://mlsanity.com/paper/2102.08427/) (2021)
- [SENT: Sentence-level Distant Relation Extraction via Negative Training](https://mlsanity.com/paper/2106.11566/) (2021)
- [PADDLES: Phase-Amplitude Spectrum Disentangled Early Stopping for Learning with Noisy Labels](https://mlsanity.com/paper/2212.03462/) (2022)
- [Instance-dependent Label Distribution Estimation for Learning with Label Noise](https://mlsanity.com/paper/2212.08380/) (2022)
- [AIO2: Online Correction of Object Labels for Deep Learning with Incomplete Annotation in Remote Sensing Image Segmentation](https://mlsanity.com/paper/2403.01641/) (2024)
- [CCMN: A General Framework for Learning with Class-Conditional Multi-Label Noise](https://mlsanity.com/paper/2105.07338/) (2021)
- [Latent Class-Conditional Noise Model](https://mlsanity.com/paper/2302.09595/) (2023)
- [NLPrompt: Noise-Label Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2412.01256/) (2024)
- [Sample Prior Guided Robust Model Learning to Suppress Noisy Labels](https://mlsanity.com/paper/2112.01197/) (2021)

## Abstract (arXiv)

Deep neural networks (DNNs) have great expressive power, which can even memorize samples with wrong labels. It is vitally important to reiterate robustness and generalization in DNNs against label corruption. To this end, this paper studies the 0-1 loss, which has a monotonic relationship with an empirical adversary (reweighted) risk~citephu2016does. Although the 0-1 loss has some robust properties, it is difficult to optimize. To efficiently optimize the 0-1 loss while keeping its robust properties, we propose a very simple and efficient loss, i.e. curriculum loss (CL). Our CL is a tighter upper bound of the 0-1 loss compared with conventional summation based surrogate losses. Moreover, CL can adaptively select samples for model training. As a result, our loss can be deemed as a novel perspective of curriculum sample selection strategy, which bridges a connection between curriculum learning and robust learning. Experimental results on benchmark datasets validate the robustness of the proposed loss.