mlsanityData to 14 Oct 2026

The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networks

arXiv 2303.01456 ↗First posted 2 Mar 2023cs.LG stat.MLcited by 23

Not on the map. Why?

Abstract (arXiv)

In this work, we study the implications of the implicit bias of gradient flow on generalization and adversarial robustness in ReLU networks. We focus on a setting where the data consists of clusters and the correlations between cluster means are small, and show that in two-layer ReLU networks gradient flow is biased towards solutions that generalize well, but are highly vulnerable to adversarial examples. Our results hold even in cases where the network has many more parameters than training examples. Despite the potential for harmful overfitting in such overparameterized settings, we prove that the implicit bias of gradient flow prevents it. However, the implicit bias also leads to non-robust solutions (susceptible to small adversarial ℓ2-perturbations), even though robust networks that fit the data exist.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 53 of its 63 references that are in the map. First posted on arXiv 2 Mar 2023.

Builds on

53 of 63 references.

References · 53

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  5. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  6. 2016Adversarial Machine Learning at Scalecited by 936
  7. 2016Practical Black-Box Attacks against Machine Learningcited by 903
  8. 2016Universal adversarial perturbationscited by 784
  9. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  10. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 678
  11. 2017Exploring Generalization in Deep Learningcited by 600
  12. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
38 more
  1. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  2. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 536
  3. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 460
  4. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 351
  5. 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 320
  6. 2018Adversarially Robust Generalization Requires More Datacited by 318
  7. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  8. 2018Audio Adversarial Examples: Targeted Attacks on Speech-to-Textcited by 274
  9. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 268
  10. 2020Directional convergence and alignment in deep learningcited by 178
  11. 2019Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectorscited by 155
  12. 2021A Universal Law of Robustness via Isoperimetrycited by 151
  13. 2022Reconstructing Training Data from Trained Neural Networkscited by 133
  14. 2018Are adversarial examples inevitable?cited by 113
  15. 2020Feature Purification: How Adversarial Training Performs Robust Deep Learningcited by 107
  16. 2022Benign Overfitting in Two-layer Convolutional Neural Networkscited by 91
  17. 2022On the Implicit Bias in Deep-Learning Algorithmscited by 86
  18. 2021Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Biascited by 83
  19. 2020Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regimecited by 81
  20. 2018Adversarial examples from computational constraintscited by 80
  21. 2018Adversarial vulnerability for any classifiercited by 78
  22. 2022Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Datacited by 74
  23. 2021A Survey On Universal Adversarial Attackcited by 59
  24. 2020The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networkscited by 58
  25. 2020A law of robustness for two-layers neural networkscited by 55
  26. 2022Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Datacited by 46
  27. 2018On the Geometry of Adversarial Examplescited by 43
  28. 2021The Dimpled Manifold Model of Adversarial Examples in Machine Learningcited by 41
  29. 2021On Margin Maximization in Linear and ReLU Networkscited by 34
  30. 2019A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distancecited by 32
  31. 2019The generalization error of max-margin linear classifiers: Benign overfitting and high dimensional asymptotics in the overparametrized regimecited by 28
  32. 2022Gradient Methods Provably Converge to Non-Robust Networkscited by 27
  33. 2021A single gradient step finds adversarial examples on random two-layers neural networkscited by 26
  34. 2021Adversarial Examples in Multi-Layer Random ReLU Networkscited by 26
  35. 2022The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networkscited by 25
  36. 2022On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Biascited by 21
  37. 2021Towards Understanding Learning in Neural Networks with Linear Teacherscited by 19
  38. 2021Shift Invariance Can Reduce Adversarial Robustnesscited by 16

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 12; 2025: 1; 2026: 3.

23 papers.

Sort:

By area:

  1. 2023Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputscited by 16
  2. 2024Trained Transformer Classifiers Generalize and Exhibit Benign Overfitting In-Contextcited by 15
  3. 2023Provable Guarantees for Neural Networks via Gradient Feature Learningcited by 14
  4. 2024Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Datacited by 12
  5. 2026The Implicit Bias of Adam and Muon on Smooth Homogeneous Neural Networkscited by 11
  6. 2024Understanding In-Context Learning of Linear Models in Transformers Through an Adversarial Lenscited by 8
  7. 2024Improving Shift Invariance in Convolutional Neural Networks with Translation Invariant Polyphase Samplingcited by 6
  8. 2024Effect of Ambient-Intrinsic Dimension Gap on Adversarial Vulnerabilitycited by 5
  9. 2024Can Implicit Bias Imply Adversarial Robustness?cited by 5
  10. 2024The Price of Implicit Bias in Adversarially Robust Generalizationcited by 5
  11. 2024Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial Statescited by 4
  12. 2024When Are Bias-Free ReLU Networks Effectively Linear Networks?cited by 4
11 more
  1. 2024Provable Privacy Attacks on Trained Shallow Neural Networkscited by 4
  2. 2023The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learningcited by 2
  3. 2023Upper and lower bounds for the Lipschitz constant of random neural networkscited by 2
  4. 2025The Features at Convergence Theorem: a first-principles alternative to the Neural Feature Ansatz for how networks learn representationscited by 2
  5. 2023Is Certifying ℓp Robustness Still Worthwhile?cited by 1
  6. 2024MALT Powers Up Adversarial Attackscited by 1
  7. 2026A Provable Energy-Guided Test-Time Defense Boosting Adversarial Robustness of Large Vision-Language Modelscited by 1
  8. 2023Frequency maps reveal the correlation between Adversarial Attacks and Implicit Biascited by 0
  9. 2023Optimization dependent generalization bound for ReLU networks based on sensitivity in the tangent bundlecited by 0
  10. 2024Adversarial Vulnerability as a Consequence of On-Manifold Inseparibilitycited by 0
  11. 2026On the Rate of Convergence of GD in Non-linear Neural Networks: An Adversarial Robustness Perspectivecited by 0