Towards Million-Scale Adversarial Robustness Evaluation With Stronger Individual Attacks
Not on the map. Why?
Abstract (arXiv)
As deep learning models are increasingly deployed in safety-critical applications, evaluating their vulnerabilities to adversarial perturbations is essential for ensuring their reliability and trustworthiness. Over the past decade, a large number of white-box adversarial robustness evaluation methods (i.e., attacks) have been proposed, ranging from single-step to multi-step methods and from individual to ensemble methods. Despite these advances, challenges remain in conducting meaningful and comprehensive robustness evaluations, particularly when it comes to large-scale testing and ensuring evaluations reflect real-world adversarial risks. In this work, we focus on image classification models and propose a novel individual attack method, Probability Margin Attack (PMA), which defines the adversarial margin in the probability space rather than the logits space. We analyze the relationship between PMA and existing cross-entropy or logits-margin-based attacks, and show that PMA can outperform the current state-of-the-art individual methods. Building on PMA, we propose two types of ensemble attacks that balance effectiveness and efficiency. Furthermore, we create a million-scale dataset, CC1M, derived from the existing CC3M dataset, and use it to conduct the first million-scale white-box adversarial robustness evaluation of adversarially-trained ImageNet models. Our findings provide valuable insights into the robustness gaps between individual versus ensemble attacks and small-scale versus million-scale evaluations.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 39 of its 45 references that are in the map. First posted on arXiv 20 Nov 2024.
Builds on
39 of 45 references.
References · 39
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2022A ConvNet for the 2020scited by 3,545
- 2016Wide Residual Networkscited by 3,057
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2016Adversarial examples in the physical worldcited by 1,762
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
27 more
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
- 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
- 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
- 2021XCiT: Cross-Covariance Image Transformerscited by 335
- 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
- 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
- 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
- 2021Improving Robustness using Generated Datacited by 233
- 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
- 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
- 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 109
- 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
- 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
- 2022A Light Recipe to Train Robust Vision Transformerscited by 64
- 2023Decoupled Kullback-Leibler Divergence Losscited by 58
- 2023Robust Principles: Architectural Design Principles for Adversarially Robust CNNscited by 46
- 2019An Alternative Surrogate Loss for PGD-based Adversarial Testingcited by 45
- 2022Practical Evaluation of Adversarial Robustness via Adaptive Auto Attackcited by 44
- 2022Revisiting Residual Networks for Adversarial Robustness: An Architectural Perspectivecited by 28
- 2021LAFEAT: Piercing Through Adversarial Defenses with Latent Featurescited by 24
- 2023Improving the Accuracy-Robustness Trade-Off of Classifiers via Adaptive Smoothingcited by 18
- 2024MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifierscited by 16
- 2020Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustnesscited by 11
- 2022Diversified Adversarial Attacks based on Conjugate Gradient Methodcited by 11
- 2024LDReg: Local Dimensionality Regularized Self-Supervised Learningcited by 11
- 2022COLLIDER: A Robust Training Framework for Backdoor Datacited by 6
- 2022Alternating Objectives Generates Stronger PGD-Based Adversarial Attackscited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.