mlsanityData to 7 Oct 2026

You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle

arXiv 1905.00877 ↗First posted 2 May 2019stat.ML cs.LG math.OCcited by 181

Not on the map. Why?

Abstract (arXiv)

Deep learning achieves state-of-the-art results in many tasks in computer vision and natural language processing. However, recent works have shown that deep networks can be vulnerable to adversarial perturbations, which raised a serious robustness issue of deep networks. Adversarial training, typically formulated as a robust optimization problem, is an effective way of improving the robustness of deep networks. A major drawback of existing adversarial training algorithms is the computational overhead of the generation of adversarial examples, typically far greater than that of the network training. This leads to the unbearable overall computational cost of adversarial training. In this paper, we show that adversarial training can be cast as a discrete time differential game. Through analyzing the Pontryagin's Maximal Principle (PMP) of the problem, we observe that the adversary update is only coupled with the parameters of the first layer of the network. This inspires us to restrict most of the forward and back propagation within the first layer of the network during adversary updates. This effectively reduces the total number of full forward and backward propagation to only one for each group of adversary updates. Therefore, we refer to this algorithm YOPO (You Only Propagate Once). Numerical experiments demonstrate that YOPO can achieve comparable defense accuracy with approximately 1/5 ~ 1/4 GPU time of the projected gradient descent (PGD) algorithm. Our codes are available at https://https://github.com/a1600012888/YOPO-You-Only-Propagate-Once.

Key facts. Cited by 181 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 67; 2022: 48; 2023: 37; 2024: 15; 2025: 10; 2026: 4.

181 papers.

Sort:

By area:

  1. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
  2. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  3. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  4. 2022Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Modelscited by 179
  5. 2021A Survey on Data Augmentation for Text Classificationcited by 111
  6. 2021AugMax: Adversarial Composition of Random Augmentations for Robust Trainingcited by 89
  7. 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 76
  8. 2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 75
  9. 2022SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustnesscited by 74
  10. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
  11. 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 65
  12. 2021HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noisecited by 61
38 more
  1. 2021A Survey On Universal Adversarial Attackcited by 59
  2. 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
  3. 2021Adversarial Robustness under Long-Tailed Distributioncited by 50
  4. 2022Practical Evaluation of Adversarial Robustness via Adaptive Auto Attackcited by 44
  5. 2022How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspectivecited by 39
  6. 2023An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learningcited by 39
  7. 2024Data Selection via Optimal Control for Language Modelscited by 38
  8. 2022Prior-Guided Adversarial Initialization for Fast Adversarial Trainingcited by 34
  9. 2021Exploring Misclassifications of Robust Neural Networks to Enhance Adversarial Attackscited by 33
  10. 2022Visual Prompting for Adversarial Robustnesscited by 33
  11. 2022DISCO: Adversarial Defense with Local Implicit Functionscited by 33
  12. 2021Attacking Adversarial Attacks as A Defensecited by 31
  13. 2023Robust Mixture-of-Expert Training for Convolutional Neural Networkscited by 30
  14. 2021Low Curvature Activations Reduce Overfitting in Adversarial Trainingcited by 29
  15. 2021On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learningcited by 29
  16. 2021Reliably fast adversarial training via latent adversarial perturbationcited by 29
  17. 2021A Review of Adversarial Attack and Defense for Classification Methodscited by 29
  18. 2021JEM++: Improved Techniques for Training JEMcited by 28
  19. 2022Robust Models are less Over-Confidentcited by 25
  20. 2021LAFEAT: Piercing Through Adversarial Defenses with Latent Featurescited by 24
  21. 2022Fast Adversarial Training with Adaptive Step Sizecited by 24
  22. 2021Towards Robust Neural Networks via Close-loop Controlcited by 23
  23. 2021Taxonomy of Machine Learning Safety: A Survey and Primercited by 22
  24. 2022TextGrad: Advancing Robustness Evaluation in NLP by Gradient-Driven Optimizationcited by 21
  25. 2022CNN Filter DB: An Empirical Investigation of Trained Convolutional Filterscited by 20
  26. 2023Defending against Adversarial Audio via Diffusion Modelcited by 20
  27. 2023Supervised Adversarial Contrastive Learning for Emotion Recognition in Conversationscited by 19
  28. 2023Targeted Attack Improves Protection against Unauthorized Diffusion Customizationcited by 18
  29. 2024Perturbing Attention Gives You More Bang for the Buck: Subtle Imaging Perturbations That Efficiently Fool Customized Diffusion Modelscited by 18
  30. 2022Improving Contrastive Learning of Sentence Embeddings with Case-Augmented Positives and Retrieved Negativescited by 16
  31. 2021Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attackcited by 15
  32. 2021Two Coupled Rejection Metrics Can Tell Adversarial Examples Apartcited by 14
  33. 2023Defenses in Adversarial Machine Learning: A Surveycited by 14
  34. 2021Universal Adversarial Training with Class-Wise Perturbationscited by 13
  35. 2022Adversarial Contrastive Learning via Asymmetric InfoNCEcited by 13
  36. 2022GA-SAM: Gradient-Strength based Adaptive Sharpness-Aware Minimization for Improved Generalizationcited by 13
  37. 2021Learning Diverse-Structured Networks for Adversarial Robustnesscited by 12
  38. 2024Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspectivecited by 12

50 most cited of 181 All 181 (JSON)