mlsanityData to 7 Oct 2026

Adversarial Robustness is at Odds with Lazy Training

arXiv 2207.00411 ↗First posted 18 Jun 2022cs.LG cs.CR stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

Recent works show that adversarial examples exist for random neural networks [Daniely and Schacham, 2020] and that these examples can be found using a single step of gradient ascent [Bubeck et al., 2021]. In this work, we extend this line of work to "lazy training" of neural networks -- a dominant model in deep learning theory in which neural networks are provably efficiently learnable. We show that over-parametrized neural networks that are guaranteed to generalize well and enjoy strong computational guarantees remain vulnerable to attacks generated using a single step of gradient ascent.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 24 of its 30 references that are in the map. First posted on arXiv 18 Jun 2022.

Builds on

24 of 30 references.

References · 24

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  5. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  6. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  7. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  8. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  9. 2018On Lazy Training in Differentiable Programmingcited by 671
  10. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  11. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  12. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
12 more
  1. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  2. 2019On Evaluating Adversarial Robustnesscited by 442
  3. 2019Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networkscited by 120
  4. 2019How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?cited by 84
  5. 2018Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hidingcited by 83
  6. 2019Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networkscited by 60
  7. 2019Convergence of Adversarial Training in Overparametrized Neural Networkscited by 58
  8. 2019A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distancecited by 32
  9. 2022Gradient Methods Provably Converge to Non-Robust Networkscited by 27
  10. 2021A single gradient step finds adversarial examples on random two-layers neural networkscited by 26
  11. 2021Adversarial Examples in Multi-Layer Random ReLU Networkscited by 26
  12. 2018Rogue Signs: Deceiving Traffic Sign Recognition with Malicious Ads and Logoscited by 14

Built on by

Cited by year

Citing papers per year: 2023: 6.

6 papers.

  1. 2023The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networkscited by 23
  2. 2023Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernelscited by 13
  3. 2023Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Linear Subspacescited by 8
  4. 2023Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approachcited by 8
  5. 2023Upper and lower bounds for the Lipschitz constant of random neural networkscited by 2
  6. 2023Fine-Grained ImageNet Classification in the Wildcited by 0