Semi-Implicit Hybrid Gradient Methods with Application to Adversarial Robustness
Not on the map. Why?
Abstract (arXiv)
Adversarial examples, crafted by adding imperceptible perturbations to natural inputs, can easily fool deep neural networks (DNNs). One of the most successful methods for training adversarially robust DNNs is solving a nonconvex-nonconcave minimax problem with an adversarial training (AT) algorithm. However, among the many AT algorithms, only Dynamic AT (DAT) and You Only Propagate Once (YOPO) guarantee convergence to a stationary point. In this work, we generalize the stochastic primal-dual hybrid gradient algorithm to develop semi-implicit hybrid gradient methods (SI-HGs) for finding stationary points of nonconvex-nonconcave minimax problems. SI-HGs have the convergence rate O(1/K), which improves upon the rate O(1/K1/2) of DAT and YOPO. We devise a practical variant of SI-HGs, and show that it outperforms other AT algorithms in terms of convergence speed and robustness.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 49 references that are in the map. First posted on arXiv 21 Feb 2022.
Builds on
34 of 49 references.
References · 34
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2016Identity Mappings in Deep Residual Networkscited by 1,960
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
- 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
- 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
- 2020Fast is better than free: Revisiting adversarial trainingcited by 723
- 2017Training GANs with Optimismcited by 217
- 2020Understanding and Improving Fast Adversarial Trainingcited by 213
- 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 181
- 2019Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methodscited by 148
22 more
- 2018A Variational Inequality Perspective on Generative Adversarial Networkscited by 144
- 2018Optimistic mirror descent in saddle-point problems: Going the extra (gradient) milecited by 132
- 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
- 2020Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimizationcited by 94
- 2019Efficient Algorithms for Smooth Minimax Optimizationcited by 75
- 2020Opportunities and Challenges in Deep Learning Adversarial Robustness: A Surveycited by 71
- 2019Reducing Noise in GAN Training with Variance Reduced Extragradientcited by 61
- 2019Convergence of Adversarial Training in Overparametrized Neural Networkscited by 58
- 2018Golden Ratio Algorithms for Variational Inequalitiescited by 38
- 2020Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionalitycited by 36
- 2018First-order Convergence Theory for Weakly-Convex-Weakly-Concave Min-max Problemscited by 33
- 2013On the Convergence Properties of Non-Euclidean Extragradient Methods for Variational Inequalities with Generalized Monotone Operatorscited by 29
- 2017Stochastic Primal-Dual Hybrid Gradient Algorithm with Arbitrary Sampling and Imaging Applicationscited by 29
- 2021Optimistic Dual Extrapolation for Coherent Non-monotone Variational Inequalitiescited by 29
- 2020On the Generalization Properties of Adversarial Trainingcited by 23
- 2017A New Randomized Block-Coordinate Primal-Dual Proximal Algorithm for Distributed Optimizationcited by 12
- 2015A Coordinate Descent Primal-Dual Algorithm with Large Step Size and Possibly Non Separable Functionscited by 11
- 2018Trust Region Based Adversarial Attack on Neural Networkscited by 11
- 2020Solving Non-Convex Non-Differentiable Min-Max Games using Proximal Gradient Methodcited by 10
- 2020Random extrapolation for primal-dual coordinate descentcited by 10
- 2020Robust Deep Learning as Optimal Control: Insights and Convergence Guaranteescited by 6
- 2019A principled approach for generating adversarial images under non-smooth dissimilarity metricscited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.