# Limited Gradient Descent: Learning With Noisy Labels

arXiv: https://arxiv.org/abs/1811.08117 · HTML: https://mlsanity.com/paper/1811.08117/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2018.

## Built on by (most cited first, 4 of 4)

- [Hard Sample Aware Noise Robust Learning for Histopathology Image Classification](https://mlsanity.com/paper/2112.03694/) (2021)
- [Sample Prior Guided Robust Model Learning to Suppress Noisy Labels](https://mlsanity.com/paper/2112.01197/) (2021)
- [Advances in Electron Microscopy with Deep Learning](https://mlsanity.com/paper/2101.01178/) (2021)
- [Generation and Analysis of Feature-Dependent Pseudo Noise for Training Deep Neural Networks](https://mlsanity.com/paper/2105.10796/) (2021)

## Abstract (arXiv)

Label noise may affect the generalization of classifiers, and the effective learning of main patterns from samples with noisy labels is an important challenge. Recent studies have shown that deep neural networks tend to prioritize the learning of simple patterns over the memorization of noise patterns. This suggests a possible method to search for the best generalization that learns the main pattern until the noise begins to be memorized. Traditional approaches often employ a clean validation set to find the best stop timing of learning, i.e., early stopping. However, the generalization performance of such methods relies on the quality of validation sets. Further, in practice, a clean validation set is sometimes difficult to obtain. To solve this problem, we propose a method that can estimate the optimal stopping timing without a clean validation set, called limited gradient descent. We modified the labels of a few samples in a noisy dataset to obtain false labels and to create a reverse pattern. By monitoring the learning progress of the noisy and reverse samples, we can determine the stop timing of learning. In this paper, we also theoretically provide some necessary conditions on learning with noisy labels. Experimental results on CIFAR-10 and CIFAR-100 datasets demonstrate that our approach has a comparable generalization performance to methods relying on a clean validation set. Thus, on the noisy Clothing-1M dataset, our approach surpasses methods that rely on a clean validation set.