Adversarial Robustness Overestimation and Instability in TRADES
Not on the map. Why?
Abstract (arXiv)
This paper examines the phenomenon of probabilistic robustness overestimation in TRADES, a prominent adversarial training method. Our study reveals that TRADES sometimes yields disproportionately high PGD validation accuracy compared to the AutoAttack testing accuracy in the multiclass classification task. This discrepancy highlights a significant overestimation of robustness for these instances, potentially linked to gradient masking. We further analyze the parameters contributing to unstable models that lead to overestimation. Our findings indicate that smaller batch sizes, lower beta values (which control the weight of the robust loss term in TRADES), larger learning rates, and higher class complexity (e.g., CIFAR-100 versus CIFAR-10) are associated with an increased likelihood of robustness overestimation. By examining metrics such as the First-Order Stationary Condition (FOSC), inner-maximization, and gradient information, we identify the underlying cause of this phenomenon as gradient masking and provide insights into it. Furthermore, our experiments show that certain unstable training instances may return to a state without robust overestimation, inspiring our attempts at a solution. In addition to adjusting parameter settings to reduce instability or retraining when overestimation occurs, we recommend incorporating Gaussian noise in inputs when the FOSC score exceed the threshold. This method aims to mitigate robustness overestimation of TRADES and other similar methods at its source, ensuring more reliable representation of adversarial robustness during evaluation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 43 references that are in the map. First posted on arXiv 10 Oct 2024.
Builds on
35 of 43 references.
References · 35
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2013Intriguing properties of neural networkscited by 5,155
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
- 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
- 2016Adversarial Machine Learning at Scalecited by 936
- 2018Robustness May Be at Odds with Accuracycited by 880
- 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
- 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
23 more
- 2020RobustBench: a standardized adversarial robustness benchmarkcited by 661
- 2020Overfitting in adversarially robust deep learningcited by 506
- 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
- 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 278
- 2015Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decompositioncited by 243
- 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 243
- 2018Adversarial Logit Pairingcited by 224
- 2020Bag of Tricks for Adversarial Trainingcited by 200
- 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 172
- 2021On the Convergence and Robustness of Adversarial Trainingcited by 130
- 2019Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networkscited by 60
- 2020On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Themcited by 59
- 2021Exploring Memorization in Adversarial Trainingcited by 55
- 2022Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 42
- 2020Random Smoothing Might be Unable to Certify ℓ_∞ Robustness for High-Dimensional Imagescited by 35
- 2018Evaluating and Understanding the Robustness of Adversarial Logit Pairingcited by 34
- 2023Randomized Adversarial Training via Taylor Expansioncited by 28
- 2018Logit Pairing Methods Can Fool Gradient-Based Attackscited by 27
- 2019Label Smoothing and Logit Squeezing: A Replacement for Adversarial Training?cited by 18
- 2021Gradient Masking and the Underestimated Robustness Threats of Differential Privacy in Deep Learningcited by 13
- 2020Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustnesscited by 11
- 2018Gradient Masking Causes CLEVER to Overestimate Adversarial Perturbation Sizecited by 9
- 2023Annealing Self-Distillation Rectification Improves Adversarial Trainingcited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.