mlsanityData to 14 Oct 2026

Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworks

arXiv 2310.15444 ↗First posted 24 Oct 2023cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Adversarial training has shown promise in building robust models against adversarial examples. A major drawback of adversarial training is the computational overhead introduced by the generation of adversarial examples. To overcome this limitation, adversarial training based on single-step attacks has been explored. Previous work improves the single-step adversarial training from different perspectives, e.g., sample initialization, loss regularization, and training strategy. Almost all of them treat the underlying model as a black box. In this work, we propose to exploit the interior building blocks of the model to improve efficiency. Specifically, we propose to dynamically sample lightweight subnetworks as a surrogate model during training. By doing this, both the forward and backward passes can be accelerated for efficient adversarial training. Besides, we provide theoretical analysis to show the model robustness can be improved by the single-step adversarial training with sampled subnetworks. Furthermore, we propose a novel sampling strategy where the sampling varies from layer to layer and from iteration to iteration. Compared with previous methods, our method not only reduces the training cost but also achieves better model robustness. Evaluations on a series of popular datasets demonstrate the effectiveness of the proposed FB-Better. Our code has been released at https://github.com/jiaxiaojunQAQ/FP-Better.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 59 of its 73 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

59 of 73 references.

References · 59

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  7. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  8. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  9. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  10. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  12. 2016Deep Networks with Stochastic Depthcited by 979
38 more
  1. 2017Ensemble Adversarial Training: Attacks and Defensescited by 840
  2. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  3. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  4. 2019Adversarial Training for Free!cited by 643
  5. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 599
  6. 2020Overfitting in adversarially robust deep learningcited by 506
  7. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 501
  8. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
  9. 2019Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attackscited by 442
  10. 2019Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attackscited by 355
  11. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
  12. 2021Enhancing the Transferability of Adversarial Attacks through Variance Tuningcited by 244
  13. 2020Understanding and Improving Fast Adversarial Trainingcited by 214
  14. 2020Bag of Tricks for Adversarial Trainingcited by 200
  15. 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 183
  16. 2018Stochastic Activation Pruning for Robust Adversarial Defensecited by 171
  17. 2021Towards Robust Vision Transformercited by 137
  18. 2021On the Convergence and Robustness of Adversarial Trainingcited by 130
  19. 2020Rethinking Bias-Variance Trade-off for Generalization of Neural Networkscited by 107
  20. 2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 95
  21. 2019NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networkscited by 94
  22. 2020Learnable Boundary Guided Adversarial Trainingcited by 93
  23. 2021Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferabilitycited by 84
  24. 2020Understanding Catastrophic Overfitting in Single-step Adversarial Trainingcited by 76
  25. 2021Improving Adversarial Robustness via Channel-wise Activation Suppressingcited by 76
  26. 2021Adversarial Laser Beam: Effective Physical-World Attack to DNNs in a Blinkcited by 75
  27. 2020Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalizationcited by 73
  28. 2020Guided Adversarial Attack for Evaluating and Enhancing Adversarial Defensescited by 62
  29. 2022Adversarial Training for High-Stakes Reliabilitycited by 60
  30. 2021AdvDrop: Adversarial Attack to DNNs by Dropping Informationcited by 56
  31. 2022A Large-scale Multiple-objective Method for Black-box Attack against Object Detectioncited by 55
  32. 2022Understanding Robust Overfitting of Adversarial Training and Beyondcited by 51
  33. 2021Targeted Attack against Deep Neural Networks via Flipping Limited Weight Bitscited by 49
  34. 2020Single-step Adversarial training with Dropout Schedulingcited by 41
  35. 2021Subspace Adversarial Trainingcited by 41
  36. 2020FakeLocator: Robust Localization of GAN-Based Face Manipulationscited by 40
  37. 2021Boosting Fast Adversarial Training with Learnable Adversarial Initializationcited by 40
  38. 2022Towards Understanding and Boosting Adversarial Transferability from a Distribution Perspectivecited by 37

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 1.

8 papers.

  1. 2024Multi-Pair Temporal Sentence Grounding via Multi-Thread Knowledge Transfer Networkcited by 15
  2. 2024Texture Re-scalable Universal Adversarial Perturbationcited by 9
  3. 2024Adversarial Training: A Surveycited by 6
  4. 2025Scale-Invariant Adversarial Attack against Arbitrary-scale Super-resolutioncited by 3
  5. 2025CIARD: Cyclic Iterative Adversarial Robustness Distillationcited by 1
  6. 2024Improving Fast Adversarial Training Paradigm: An Example Taxonomy Perspectivecited by 0
  7. 2025FERD: Fairness-Enhanced Data-Free Robustness Distillationcited by 0
  8. 2026Unveiling the Backdoor Mechanism Hidden Behind Catastrophic Overfitting in Fast Adversarial Trainingcited by 0