DeepMoM: Robust Deep Learning With Median-of-Means
Not on the map. Why?
Abstract (arXiv)
Data used in deep learning is notoriously problematic. For example, data are usually combined from diverse sources, rarely cleaned and vetted thoroughly, and sometimes corrupted on purpose. Intentional corruption that targets the weak spots of algorithms has been studied extensively under the label of "adversarial attacks." In contrast, the arguably much more common case of corruption that reflects the limited quality of data has been studied much less. Such "random" corruptions are due to measurement errors, unreliable sources, convenience sampling, and so forth. These kinds of corruption are common in deep learning, because data are rarely collected according to strict protocols -- in strong contrast to the formalized data collection in some parts of classical statistics. This paper concerns such corruption. We introduce an approach motivated by very recent insights into median-of-means and Le Cam's principle, we show that the approach can be readily implemented, and we demonstrate that it performs very well in practice. In conclusion, we believe that our approach is a very promising alternative to standard parameter training based on least-squares and cross-entropy loss.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 36 of its 73 references that are in the map. First posted on arXiv 28 May 2021.
Builds on
36 of 73 references.
References · 36
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2016Adversarial examples in the physical worldcited by 1,762
- 2016Adversarial Machine Learning at Scalecited by 934
- 2018Robustness May Be at Odds with Accuracycited by 873
- 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
- 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
- 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
- 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
- 2017One pixel attack for fooling deep neural networkscited by 579
- 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
- 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
24 more
- 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 459
- 2017Adversarial Examples: Attacks and Defenses for Deep Learningcited by 350
- 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
- 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 327
- 2017A General and Adaptive Robust Loss Functioncited by 236
- 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
- 2019Adversarial Training and Robustness for Multiple Perturbationscited by 182
- 2018A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspectivecited by 111
- 2017Delving into adversarial attacks on deep policiescited by 104
- 2019Optimization for deep learning: theory and algorithmscited by 70
- 2014Learning without Concentrationcited by 50
- 2016Studying Very Low Resolution Recognition Using Deep Networkscited by 36
- 2017Adversarial Example Defenses: Ensembles of Weak Defenses are not Strongcited by 32
- 2015Robust Optimization for Deep Regressioncited by 22
- 2019A Direct Approach to Robust Deep Learning Using Adversarial Networkscited by 18
- 2018AdVersarial: Perceptual Ad Blocking meets Adversarial Machine Learningcited by 14
- 2021Activation Functions in Artificial Neural Networks: A Systematic Overviewcited by 12
- 2017Regularization, sparse recovery, and median-of-means tournamentscited by 11
- 2018How Robust are Deep Neural Networks?cited by 10
- 2020Risk Bounds for Robust Deep Learningcited by 9
- 2020Is there a role for statistics in artificial intelligence?cited by 5
- 2018Towards Robust Deep Neural Networkscited by 4
- 2018On Hyperparameter Search in Cluster Ensemblescited by 3
- 2015A General Hybrid Clustering Techniquecited by 1
Built on by
Citing papers per year: 2022: 2.
2 papers.