# On the Robustness of Monte Carlo Dropout Trained with Noisy Labels

arXiv: https://arxiv.org/abs/2103.12002 · HTML: https://mlsanity.com/paper/2103.12002/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 31 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 23 of 23)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [The State of Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1902.09574/) (2019)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [Decoupling "when to update" from "how to update"](https://mlsanity.com/paper/1706.02613/) (2017)
- [Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise](https://mlsanity.com/paper/1802.05300/) (2018)
- [SELF: Learning to Filter Noisy Labels with Self-Ensembling](https://mlsanity.com/paper/1910.01842/) (2019)
- [Curriculum Loss: Robust Learning and Generalization against Label Corruption](https://mlsanity.com/paper/1905.10045/) (2019)
- [Training Sparse Neural Networks](https://mlsanity.com/paper/1611.06694/) (2016)
- [Part-dependent Label Noise: Towards Instance-dependent Label Noise](https://mlsanity.com/paper/2006.07836/) (2020)
- [A Semi-Supervised Two-Stage Approach to Learning from Noisy Labels](https://mlsanity.com/paper/1802.02679/) (2018)
- [Label Noise Types and Their Effects on Deep Learning](https://mlsanity.com/paper/2003.10471/) (2020)
- [Uncertainty Based Detection and Relabeling of Noisy Image Labels](https://mlsanity.com/paper/1906.11876/) (2019)
- [Robust Deep Learning with Active Noise Cancellation for Spatial Computing](https://mlsanity.com/paper/2011.08341/) (2020)

## Built on by (most cited first, 11 of 11)

- [Sparse Double Descent: Where Network Pruning Aggravates Overfitting](https://mlsanity.com/paper/2206.08684/) (2022)
- [Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition](https://mlsanity.com/paper/2209.00641/) (2022)
- [Improving evidential deep learning via multi-task learning](https://mlsanity.com/paper/2112.09368/) (2021)
- [With a Little Push, NLI Models can Robustly and Efficiently Predict Faithfulness](https://mlsanity.com/paper/2305.16819/) (2023)
- [Maximum Likelihood Uncertainty Estimation: Robustness to Outliers](https://mlsanity.com/paper/2202.03870/) (2022)
- [Unveiling the Potential of Knowledge-Prompted ChatGPT for Enhancing Drug Trafficking Detection on Social Media](https://mlsanity.com/paper/2307.03699/) (2023)
- [Logistic-Normal Likelihoods for Heteroscedastic Label Noise](https://mlsanity.com/paper/2304.02849/) (2023)
- [Data-Driven Prediction and Uncertainty Quantification of PWR Crud-Induced Power Shift Using Convolutional Neural Networks](https://mlsanity.com/paper/2407.04726/) (2024)
- [Deep Learning with Label Noise: A Hierarchical Approach](https://mlsanity.com/paper/2205.14299/) (2022)
- [Bayesian uncertainty-aware deep learning with noisy labels: Tackling annotation ambiguity in EEG seizure detection](https://mlsanity.com/paper/2410.19815/) (2024)
- [Stabilizing Information Flow Entropy: Regularization for Safe and Interpretable Autonomous Driving Perception](https://mlsanity.com/paper/2509.16277/) (2025)

## Abstract (arXiv)

The memorization effect of deep learning hinders its performance to effectively generalize on test set when learning with noisy labels. Prior study has discovered that epistemic uncertainty techniques are robust when trained with noisy labels compared with neural networks without uncertainty estimation. They obtain prolonged memorization effect and better generalization performance under the adversarial setting of noisy labels. Due to its superior performance amongst other selected epistemic uncertainty methods under noisy labels, we focus on Monte Carlo Dropout (MCDropout) and investigate why it is robust when trained with noisy labels. Through empirical studies on datasets MNIST, CIFAR-10, Animal-10n, we deep dive into three aspects of MCDropout under noisy label setting: 1. efficacy: understanding the learning behavior and test accuracy of MCDropout when training set contains artificially generated or naturally embedded label noise; 2. representation volatility: studying the responsiveness of neurons by examining the mean and standard deviation on each neuron's activation; 3. network sparsity: investigating the network support of MCDropout in comparison with deterministic neural networks. Our findings suggest that MCDropout further sparsifies and regularizes the deterministic neural networks and thus provides higher robustness against noisy labels.