# Learning Deep Networks from Noisy Labels with Dropout Regularization

arXiv: https://arxiv.org/abs/1705.03419 · HTML: https://mlsanity.com/paper/1705.03419/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2017.

## Built on by (most cited first, 30 of 32)

- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Sample Selection for Fair and Robust Training](https://mlsanity.com/paper/2110.14222/) (2021)
- [On Tilted Losses in Machine Learning: Theory and Applications](https://mlsanity.com/paper/2109.06141/) (2021)
- [Characterizing Datapoints via Second-Split Forgetting](https://mlsanity.com/paper/2210.15031/) (2022)
- [Joint Negative and Positive Learning for Noisy Labels](https://mlsanity.com/paper/2104.06574/) (2021)
- [Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection](https://mlsanity.com/paper/2111.14932/) (2021)
- [Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond](https://mlsanity.com/paper/2104.01493/) (2021)
- [Investigating the Learning Behaviour of In-context Learning: A Comparison with Supervised Learning](https://mlsanity.com/paper/2307.15411/) (2023)
- [SAP: Corrective Machine Unlearning with Scaled Activation Projection for Label Noise Robustness](https://mlsanity.com/paper/2403.08618/) (2024)
- [Noisy Annotation Refinement for Object Detection](https://mlsanity.com/paper/2110.10456/) (2021)
- [Learning Discriminative Dynamics with Label Corruption for Noisy Label Detection](https://mlsanity.com/paper/2405.19902/) (2024)
- [Designing and Training of Lightweight Neural Networks on Edge Devices using Early Halting in Knowledge Distillation](https://mlsanity.com/paper/2209.15560/) (2022)
- [RC-Mixup: A Data Augmentation Strategy against Noisy Data for Regression Tasks](https://mlsanity.com/paper/2405.17938/) (2024)
- [Policy Learning from Large Vision-Language Model Feedback without Reward Modeling](https://mlsanity.com/paper/2507.23391/) (2025)
- [On the Evaluation of Sequential Machine Learning for Network Intrusion Detection](https://mlsanity.com/paper/2106.07961/) (2021)
- [Multi-class Label Noise Learning via Loss Decomposition and Centroid Estimation](https://mlsanity.com/paper/2203.10858/) (2022)
- [Robust Product Classification with Instance-Dependent Noise](https://mlsanity.com/paper/2209.06946/) (2022)
- [Real-time Adaptation for Condition Monitoring Signal Prediction using Label-aware Neural Processes](https://mlsanity.com/paper/2403.16377/) (2024)
- [Training Classifiers that are Universally Robust to All Label Noise Levels](https://mlsanity.com/paper/2105.13892/) (2021)
- [Rethinking Noisy Label Learning in Real-world Annotation Scenarios from the Noise-type Perspective](https://mlsanity.com/paper/2307.16889/) (2023)
- [FS-Net: Full Scale Network and Adaptive Threshold for Improving Extraction of Micro-Retinal Vessel Structures](https://mlsanity.com/paper/2311.08059/) (2023)
- [AlleNoise: large-scale text classification benchmark dataset with real-world label noise](https://mlsanity.com/paper/2407.10992/) (2024)
- [Granular-ball Representation Learning for Deep CNN on Learning with Label Noise](https://mlsanity.com/paper/2409.03254/) (2024)
- [A Novel Perspective for Positive-Unlabeled Learning via Noisy Labels](https://mlsanity.com/paper/2103.04685/) (2021)
- [Understanding and Mitigating the Bias in Sample Selection for Learning with Noisy Labels](https://mlsanity.com/paper/2401.13360/) (2024)
- [NoisyAG-News: A Benchmark for Addressing Instance-Dependent Noise in Text Classification](https://mlsanity.com/paper/2407.06579/) (2024)
- [Training Gradient Boosted Decision Trees on Tabular Data Containing Label Noise for Classification Tasks](https://mlsanity.com/paper/2409.08647/) (2024)
- [Hypothesis Testing for Class-Conditional Label Noise](https://mlsanity.com/paper/2103.02630/) (2021)
- [A robust approach for deep neural networks in presence of label noise: relabelling and filtering instances during training](https://mlsanity.com/paper/2109.03748/) (2021)
- ["Is not the truth the truth?": Analyzing the Impact of User Validations for Bus In/Out Detection in Smartphone-based Surveys](https://mlsanity.com/paper/2202.11961/) (2022)

## Abstract (arXiv)

Large datasets often have unreliable labels-such as those obtained from Amazon's Mechanical Turk or social media platforms-and classifiers trained on mislabeled datasets often exhibit poor performance. We present a simple, effective technique for accounting for label noise when training deep neural networks. We augment a standard deep network with a softmax layer that models the label noise statistics. Then, we train the deep network and noise model jointly via end-to-end stochastic gradient descent on the (perhaps mislabeled) dataset. The augmented model is overdetermined, so in order to encourage the learning of a non-trivial noise model, we apply dropout regularization to the weights of the noise model during training. Numerical experiments on noisy versions of the CIFAR-10 and MNIST datasets show that the proposed dropout technique outperforms state-of-the-art methods.