mlsanityData to 7 Oct 2026

A single gradient step finds adversarial examples on random two-layers neural networks

arXiv 2104.03863 ↗First posted 8 Apr 2021cs.LG cs.CR stat.MLcited by 26

Not on the map. Why?

Abstract (arXiv)

Daniely and Schacham recently showed that gradient descent finds adversarial examples on random undercomplete two-layers ReLU neural networks. The term "undercomplete" refers to the fact that their proof only holds when the number of neurons is a vanishing fraction of the ambient dimension. We extend their result to the overcomplete case, where the number of neurons is larger than the dimension (yet also subexponential in the dimension). In fact we prove that a single step of gradient descent suffices. We also show this result for any subexponential width random neural network with smooth activation function.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 12 of its 20 references that are in the map. First posted on arXiv 8 Apr 2021.

Builds on

12 of 20 references.

References · 12

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  5. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  6. 2016Universal adversarial perturbationscited by 774
  7. 2018Robustness via curvature regularization, and vice versacited by 174
  8. 2019Adversarial Robustness through Local Linearizationcited by 146
  9. 2020Feature Purification: How Adversarial Training Performs Robust Deep Learningcited by 107
  10. 2019A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distancecited by 32
  11. 2021Non-asymptotic approximations of neural networks by Gaussian processescited by 18
  12. 2020Most ReLU Networks Suffer from ℓ2 Adversarial Perturbationscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 10; 2023: 8; 2024: 3; 2025: 3; 2026: 1.

26 papers.

Sort:

By area:

  1. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  2. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 98
  3. 2022Gradient Methods Provably Converge to Non-Robust Networkscited by 27
  4. 2021Adversarial Examples in Multi-Layer Random ReLU Networkscited by 26
  5. 2023The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networkscited by 23
  6. 2022Evolution of Neural Tangent Kernels under Benign and Adversarial Trainingcited by 14
  7. 2022Adversarial Examples in Random Neural Networks with General Activationscited by 13
  8. 2025Unveiling the Basin-Like Loss Landscape in Large Language Modelscited by 13
  9. 2024Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Datacited by 12
  10. 2022On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimescited by 10
  11. 2022Adversarial Reprogramming Revisitedcited by 10
  12. 2023Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Linear Subspacescited by 8
14 more
  1. 2022Adversarial Robustness is at Odds with Lazy Trainingcited by 6
  2. 2023How many dimensions are required to find an adversarial example?cited by 5
  3. 2024Effect of Ambient-Intrinsic Dimension Gap on Adversarial Vulnerabilitycited by 5
  4. 2022Origins of Low-dimensional Adversarial Perturbationscited by 3
  5. 2023Upper and lower bounds for the Lipschitz constant of random neural networkscited by 2
  6. 2022Adversarial Noises Are Linearly Separable for (Nearly) Random Neural Networkscited by 1
  7. 2022Blessings and Curses of Covariate Shifts: Adversarial Learning Dynamics, Directional Convergence, and Equilibriacited by 1
  8. 2023A generalizable framework for low-rank tensor completion with numerical priorscited by 1
  9. 2024MALT Powers Up Adversarial Attackscited by 1
  10. 2026A Provable Energy-Guided Test-Time Defense Boosting Adversarial Robustness of Large Vision-Language Modelscited by 1
  11. 2022mathscrH-Consistency Estimation Error of Surrogate Loss Minimizerscited by 0
  12. 2023A Linearly Convergent GAN Inversion-based Algorithm for Reverse Engineering of Deceptionscited by 0
  13. 2025Deep Neural Nets as Hamiltonianscited by 0
  14. 2025Existence of Adversarial Examples for Random Convolutional Networks via Isoperimetric Inequalities on mathbbso(d)cited by 0