mlsanityData to 7 Oct 2026

Adversarial Training for Gradient Descent: Analysis Through its Continuous-time Approximation

arXiv 2105.08037 ↗First posted 17 May 2021cs.LG math.OC math.PRcited by 3

Not on the map. Why?

Abstract (arXiv)

Adversarial training has gained great popularity as one of the most effective defenses for deep neural network and more generally for gradient-based machine learning models against adversarial perturbations on data points. This paper establishes a continuous-time approximation for the mini-max game of adversarial training. This approximation approach allows for precise and analytical comparisons between stochastic gradient descent and its adversarial training counterpart; and confirms theoretically the robustness of adversarial training from a new gradient-flow viewpoint. The analysis is then corroborated through various analytical and numerical examples.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 26 of its 46 references that are in the map. First posted on arXiv 17 May 2021.

Builds on

26 of 46 references.

References · 26

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  5. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  6. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 333
  7. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 285
  8. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 242
  9. 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 181
  10. 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 148
  11. 2015Stochastic modified equations and adaptive stochastic gradient algorithmscited by 141
  12. 2018Rademacher Complexity for Adversarially Robust Generalizationcited by 132
14 more
  1. 2018Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundationscited by 109
  2. 2017Stochastic gradient descent performs variational inference, converges to limit cycles for deep networkscited by 108
  3. 2019Adversarial Robustness Against the Union of Multiple Perturbation Modelscited by 88
  4. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  5. 2019Convergence of Adversarial Training in Overparametrized Neural Networkscited by 58
  6. 2018Deep Defense: Training DNNs with Improved Adversarial Robustnesscited by 41
  7. 2020Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionalitycited by 36
  8. 2017Deep Relaxation: partial differential equations for optimizing deep neural networkscited by 27
  9. 2020More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Modelscited by 27
  10. 2018Distributionally Robust Mean-Variance Portfolio Selection with Wasserstein Distancescited by 24
  11. 2020Distributionally Robust Batch Contextual Banditscited by 20
  12. 2020Towards Understanding the Dynamics of the First-Order Adversariescited by 7
  13. 2020Robust Deep Learning as Optimal Control: Insights and Convergence Guaranteescited by 6
  14. 2020Distributionally Robust Parametric Maximum Likelihood Estimationcited by 3

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 2.

3 papers.

  1. 2025Unbiased and Sign Compression in Distributed Learning: Comparing Noise Resilience via SDEscited by 4
  2. 2023Unlocking optimal batch size schedules using continuous-time control and perturbation theorycited by 3
  3. 2025Accelerated Gradient Methods with Biased Gradient Estimates: Risk Sensitivity, High-Probability Guarantees, and Large Deviation Boundscited by 1