mlsanityData to 7 Oct 2026

Dropout can Simulate Exponential Number of Models for Sample Selection Techniques

arXiv 2202.13203 ↗First posted 26 Feb 2022cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Following Coteaching, generally in the literature, two models are used in sample selection based approaches for training with noisy labels. Meanwhile, it is also well known that Dropout when present in a network trains an ensemble of sub-networks. We show how to leverage this property of Dropout to train an exponential number of shared models, by training a single model with Dropout. We show how we can modify existing two model-based sample selection methodologies to use an exponential number of shared models. Not only is it more convenient to use a single model with Dropout, but this approach also combines the natural benefits of Dropout with that of training an exponential number of models, leading to improved results.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 39 references that are in the map. First posted on arXiv 26 Feb 2022.

Builds on

26 of 39 references.

References · 26

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  3. 2017A Closer Look at Memorization in Deep Networkscited by 936
  4. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
  5. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  6. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  7. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  8. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
  9. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
  10. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 411
  11. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  12. 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
14 more
  1. 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
  2. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  3. 2018Dimensionality-Driven Learning with Noisy Labelscited by 173
  4. 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
  5. 2017Learning from Noisy Labels with Distillationcited by 160
  6. 2020Learning with Instance-Dependent Label Noise: A Sample Sieve Approachcited by 158
  7. 2021Provably End-to-end Label-Noise Learning without Anchor Pointscited by 97
  8. 2019Curriculum Loss: Robust Learning and Generalization against Label Corruptioncited by 87
  9. 2020DMT: Dynamic Mutual Training for Semi-Supervised Learningcited by 77
  10. 2021Faster Meta Update Strategy for Noise-Robust Deep Learningcited by 38
  11. 2020EvidentialMix: Learning with Combined Open-set and Closed-set Noisy Labelscited by 28
  12. 2020Learning Not to Learn in the Presence of Noisy Labelscited by 9
  13. 2021A Framework using Contrastive Learning for Classification with Noisy Labelscited by 5
  14. 2021Transform consistency for learning with noisy labelscited by 2

Built on by

Not cited yet.