mlsanityData to 7 Oct 2026

Adversarial Examples in Multi-Layer Random ReLU Networks

arXiv 2106.12611 ↗First posted 23 Jun 2021cs.LG cs.AI stat.MLcited by 26

Not on the map. Why?

Abstract (arXiv)

We consider the phenomenon of adversarial examples in ReLU networks with independent gaussian parameters. For networks of constant depth and with a large range of widths (for instance, it suffices if the width of each layer is polynomial in that of any other layer), small perturbations of input vectors lead to large changes of outputs. This generalizes results of Daniely and Schacham (2020) for networks of rapidly decreasing width and of Bubeck et al (2021) for two-layer networks. The proof shows that adversarial examples arise in these networks because the functions that they compute are very close to linear. Bottleneck layers in the network play a key role: the minimal width up to some point in the network determines scales and sensitivities of mappings computed up to that point. The main result is for networks with constant depth, but we also show that some constraint on depth is necessary for a result of this kind, because there are suitably deep networks that, with constant probability, compute a function that is close to constant.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 12 of its 16 references that are in the map. First posted on arXiv 23 Jun 2021.

Builds on

12 of 16 references.

References · 12

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  5. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  6. 2017Detecting Adversarial Samples from Artifactscited by 244
  7. 2021A Universal Law of Robustness via Isoperimetrycited by 150
  8. 2019Adversarial Robustness through Local Linearizationcited by 146
  9. 2018Adversarial examples from computational constraintscited by 79
  10. 2019A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distancecited by 32
  11. 2021A single gradient step finds adversarial examples on random two-layers neural networkscited by 26
  12. 2020Most ReLU Networks Suffer from ℓ2 Adversarial Perturbationscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 13; 2023: 5; 2024: 3; 2025: 2; 2026: 2.

26 papers.

Sort:

By area:

  1. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  2. 2022Gradient Methods Provably Converge to Non-Robust Networkscited by 27
  3. 2021On the Existence of the Adversarial Bayes Classifier (Extended Version)cited by 25
  4. 2023The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networkscited by 23
  5. 2022Overparameterized Linear Regression under Adversarial Attackscited by 21
  6. 2022Evolution of Neural Tangent Kernels under Benign and Adversarial Trainingcited by 14
  7. 2022Adversarial Examples in Random Neural Networks with General Activationscited by 13
  8. 2024Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Datacited by 12
  9. 2022On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimescited by 10
  10. 2022Adversarial Reprogramming Revisitedcited by 10
  11. 2022Nash Equilibria and Pitfalls of Adversarial Training in Adversarial Robustness Gamescited by 8
  12. 2023Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Linear Subspacescited by 8
14 more
  1. 2022Adversarial Robustness is at Odds with Lazy Trainingcited by 6
  2. 2023How many dimensions are required to find an adversarial example?cited by 5
  3. 2022Origins of Low-dimensional Adversarial Perturbationscited by 3
  4. 2022On Trace of PGD-Like Adversarial Attackscited by 2
  5. 2023Upper and lower bounds for the Lipschitz constant of random neural networkscited by 2
  6. 2022Adversarial Noises Are Linearly Separable for (Nearly) Random Neural Networkscited by 1
  7. 2022Blessings and Curses of Covariate Shifts: Adversarial Learning Dynamics, Directional Convergence, and Equilibriacited by 1
  8. 2024Feature compression is the root cause of adversarial fragility in neural network classifierscited by 1
  9. 2024MALT Powers Up Adversarial Attackscited by 1
  10. 2026A Provable Energy-Guided Test-Time Defense Boosting Adversarial Robustness of Large Vision-Language Modelscited by 1
  11. 2022mathscrH-Consistency Estimation Error of Surrogate Loss Minimizerscited by 0
  12. 2025Deep Neural Nets as Hamiltonianscited by 0
  13. 2025Existence of Adversarial Examples for Random Convolutional Networks via Isoperimetric Inequalities on mathbbso(d)cited by 0
  14. 2026Most Convolutional Networks Suffer from Small Adversarial Perturbationscited by 0