mlsanityData to 7 Oct 2026

Gradient Methods Provably Converge to Non-Robust Networks

arXiv 2202.04347 ↗First posted 9 Feb 2022cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Despite a great deal of research, it is still unclear why neural networks are so susceptible to adversarial examples. In this work, we identify natural settings where depth-2 ReLU networks trained with gradient flow are provably non-robust (susceptible to small adversarial ℓ2-perturbations), even when robust networks that classify the training dataset correctly exist. Perhaps surprisingly, we show that the well-known implicit bias towards margin maximization induces bias towards non-robust networks, by proving that every network which satisfies the KKT conditions of the max-margin problem is non-robust.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 33 of its 40 references that are in the map. First posted on arXiv 9 Feb 2022.

Builds on

33 of 40 references.

References · 33

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  6. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  7. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  8. 2016Universal adversarial perturbationscited by 774
  9. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  10. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  11. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  12. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
21 more
  1. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 459
  2. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  3. 2018Adversarially Robust Generalization Requires More Datacited by 318
  4. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  5. 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 308
  6. 2018Audio Adversarial Examples: Targeted Attacks on Speech-to-Textcited by 268
  7. 2020Directional convergence and alignment in deep learningcited by 172
  8. 2019Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectorscited by 155
  9. 2018Are adversarial examples inevitable?cited by 113
  10. 2020Feature Purification: How Adversarial Training Performs Robust Deep Learningcited by 107
  11. 2018Adversarial examples from computational constraintscited by 79
  12. 2018Adversarial vulnerability for any classifiercited by 78
  13. 2021A Survey On Universal Adversarial Attackcited by 59
  14. 2018On the Geometry of Adversarial Examplescited by 43
  15. 2021The Dimpled Manifold Model of Adversarial Examples in Machine Learningcited by 41
  16. 2021On Margin Maximization in Linear and ReLU Networkscited by 34
  17. 2019A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distancecited by 32
  18. 2021A single gradient step finds adversarial examples on random two-layers neural networkscited by 26
  19. 2021Adversarial Examples in Multi-Layer Random ReLU Networkscited by 26
  20. 2021Shift Invariance Can Reduce Adversarial Robustnesscited by 16
  21. 2020Most ReLU Networks Suffer from ℓ2 Adversarial Perturbationscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 11; 2024: 7; 2025: 1; 2026: 2.

27 papers.

Sort:

By area:

  1. 2022On the Implicit Bias in Deep-Learning Algorithmscited by 84
  2. 2022Shortcut Learning of Large Language Models in Natural Language Understandingcited by 83
  3. 2022Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Datacited by 46
  4. 2024The Implicit Bias of Adam on Separable Datacited by 32
  5. 2023Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximizationcited by 29
  6. 2023Simplicity Bias in 1-Hidden Layer Neural Networkscited by 26
  7. 2023The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networkscited by 23
  8. 2023Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Datacited by 18
  9. 2023Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputscited by 16
  10. 2022Adversarial Examples in Random Neural Networks with General Activationscited by 13
  11. 2022Adversarial Reprogramming Revisitedcited by 10
  12. 2023Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Linear Subspacescited by 8
15 more
  1. 2022Adversarial Robustness is at Odds with Lazy Trainingcited by 6
  2. 2026The Implicit Bias of Adam and Muon on Smooth Homogeneous Neural Networkscited by 6
  3. 2023Deep ReLU Networks Have Surprisingly Simple Polytopescited by 5
  4. 2024Effect of Ambient-Intrinsic Dimension Gap on Adversarial Vulnerabilitycited by 5
  5. 2023Robust Linear Regression: Gradient-descent, Early-stopping, and Beyondcited by 4
  6. 2023Fundamental Limits of Membership Inference Attacks on Machine Learning Modelscited by 4
  7. 2024The Price of Implicit Bias in Adversarially Robust Generalizationcited by 4
  8. 2024Provable Privacy Attacks on Trained Shallow Neural Networkscited by 4
  9. 2024When Are Bias-Free ReLU Networks Effectively Linear Networks?cited by 3
  10. 2023Upper and lower bounds for the Lipschitz constant of random neural networkscited by 2
  11. 2023Is Certifying ℓp Robustness Still Worthwhile?cited by 1
  12. 2024Feature compression is the root cause of adversarial fragility in neural network classifierscited by 1
  13. 2024MALT Powers Up Adversarial Attackscited by 1
  14. 2026A Provable Energy-Guided Test-Time Defense Boosting Adversarial Robustness of Large Vision-Language Modelscited by 1
  15. 2025Existence of Adversarial Examples for Random Convolutional Networks via Isoperimetric Inequalities on mathbbso(d)cited by 0