mlsanityData to 7 Oct 2026

Rethinking PGD Attack: Is Sign Function Necessary?

arXiv 2312.01260 ↗First posted 3 Dec 2023cs.LG cs.CR stat.MLcited by 2

Not on the map. Why?

Abstract (arXiv)

Neural networks have demonstrated success in various domains, yet their performance can be significantly degraded by even a small input perturbation. Consequently, the construction of such perturbations, known as adversarial attacks, has gained significant attention, many of which fall within "white-box" scenarios where we have full access to the neural network. Existing attack algorithms, such as the projected gradient descent (PGD), commonly take the sign function on the raw gradient before updating adversarial inputs, thereby neglecting gradient magnitude information. In this paper, we present a theoretical analysis of how such sign-based update algorithm influences step-wise attack performance, as well as its caveat. We also interpret why previous attempts of directly using raw gradients failed. Based on that, we further propose a new raw gradient descent (RGD) algorithm that eliminates the use of sign. Specifically, we convert the constrained optimization problem into an unconstrained one, by introducing a new hidden variable of non-clipped perturbation that can move beyond the constraint. The effectiveness of the proposed RGD algorithm has been demonstrated extensively in experiments, outperforming PGD and other competitors in various settings, without incurring any additional computational overhead. The codes is available in https://github.com/JunjieYang97/RGD.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 34 of its 45 references that are in the map. First posted on arXiv 3 Dec 2023.

Builds on

34 of 45 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2014Going Deeper with Convolutionscited by 4,231
  8. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  9. 2016Adversarial examples in the physical worldcited by 1,762
  10. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  11. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  12. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
22 more
  1. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  2. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  3. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  4. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  5. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  6. 2020Overfitting in adversarially robust deep learningcited by 504
  7. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499
  8. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
  9. 2020Understanding and Improving Fast Adversarial Trainingcited by 213
  10. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  11. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
  12. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  13. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
  14. 2020RayS: A Ray Searching Method for Hard-label Adversarial Attackcited by 94
  15. 2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 94
  16. 2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 75
  17. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
  18. 2019advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorchcited by 69
  19. 2022A Light Recipe to Train Robust Vision Transformerscited by 64
  20. 2022Efficient and Effective Augmentation Strategy for Adversarial Trainingcited by 43
  21. 2021CausalAdv: Adversarial Robustness through the Lens of Causalitycited by 37
  22. 2022Towards Good Practices in Evaluating Transfer Adversarial Attackscited by 14

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2025Beyond Vulnerabilities: A Survey of Adversarial Attacks as Both Threats and Defenses in Computer Vision Systemscited by 4
  2. 2024Deciphering the Definition of Adversarial Robustness for post-hoc OOD Detectorscited by 2