Over-parameterization and Adversarial Robustness in Neural Networks: An Overview and Empirical Analysis
Not on the map. Why?
Abstract (arXiv)
Thanks to their extensive capacity, over-parameterized neural networks exhibit superior predictive capabilities and generalization. However, having a large parameter space is considered one of the main suspects of the neural networks' vulnerability to adversarial example -- input samples crafted ad-hoc to induce a desired misclassification. Relevant literature has claimed contradictory remarks in support of and against the robustness of over-parameterized networks. These contradictory findings might be due to the failure of the attack employed to evaluate the networks' robustness. Previous research has demonstrated that depending on the considered model, the algorithm employed to generate adversarial examples may not function properly, leading to overestimating the model's robustness. In this work, we empirically study the robustness of over-parameterized networks against adversarial examples. However, unlike the previous works, we also evaluate the considered attack's reliability to support the results' veracity. Our results show that over-parameterized networks are robust against adversarial attacks as opposed to their under-parameterized counterparts.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 42 of its 48 references that are in the map. First posted on arXiv 14 Jun 2024.
Builds on
42 of 48 references.
References · 42
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2016Understanding deep learning requires rethinking generalizationcited by 1,657
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
- 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
- 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
- 2017Evasion Attacks against Machine Learning at Test Timecited by 686
- 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
30 more
- 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
- 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
- 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
- 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 508
- 2020Overfitting in adversarially robust deep learningcited by 504
- 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
- 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
- 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
- 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
- 2021Deep learning: a statistical viewpointcited by 239
- 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 187
- 2018To understand deep learning we need to understand kernel learningcited by 186
- 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 182
- 2021A Universal Law of Robustness via Isoperimetrycited by 150
- 2018Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networkscited by 140
- 2020Rethinking Bias-Variance Trade-off for Generalization of Neural Networkscited by 107
- 2018The Singular Values of Convolutional Layerscited by 95
- 2020Do Wider Neural Networks Really Help Adversarial Robustness?cited by 79
- 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
- 2018A Modern Take on the Bias-Variance Tradeoff in Neural Networkscited by 64
- 2019Convergence of Adversarial Training in Overparametrized Neural Networkscited by 58
- 2022The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regressioncited by 40
- 2018Logit Pairing Methods Can Fool Gradient-Based Attackscited by 27
- 2021Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examplescited by 23
- 2022Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Powercited by 23
- 2022Overparameterized Linear Regression under Adversarial Attackscited by 21
- 2022Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)cited by 18
- 2019Scaling up the randomized gradient-free adversarial attack reveals overestimation of robustness using established attackscited by 11
- 2022Overparametrization improves robustness against adversarial attacks: A replication studycited by 1
- 2022Gradient Obfuscation Checklist Test Gives a False Sense of Securitycited by 1
Built on by
Citing papers per year: 2025: 2.
2 papers.