mlsanityData to 7 Oct 2026

Practical Evaluation of Adversarial Robustness via Adaptive Auto Attack

arXiv 2203.05154 ↗First posted 10 Mar 2022cs.CVcited by 44

Not on the map. Why?

Abstract (arXiv)

Defense models against adversarial attacks have grown significantly, but the lack of practical evaluation methods has hindered progress. Evaluation can be defined as looking for defense models' lower bound of robustness given a budget number of iterations and a test dataset. A practical evaluation method should be convenient (i.e., parameter-free), efficient (i.e., fewer iterations) and reliable (i.e., approaching the lower bound of robustness). Towards this target, we propose a parameter-free Adaptive Auto Attack (A3) evaluation method which addresses the efficiency and reliability in a test-time-training fashion. Specifically, by observing that adversarial examples to a specific defense model follow some regularities in their starting points, we design an Adaptive Direction Initialization strategy to speed up the evaluation. Furthermore, to approach the lower bound of robustness under the budget number of iterations, we propose an online statistics-based discarding strategy that automatically identifies and abandons hard-to-attack images. Extensive experiments demonstrate the effectiveness of our A3. Particularly, we apply A3 to nearly 50 widely-used defense models. By consuming much fewer iterations than existing methods, i.e., 1/10 on average (10× speed up), we achieve lower robust accuracy in all cases. Notably, we won first place out of 1681 teams in CVPR 2021 White-box Adversarial Attacks on Defense Models competitions with this method. Code is available at: hrefhttps://github.com/liuye6666/adaptiveautoattackhttps://github.com/liuye6666/adaptiveautoattack

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 48 of its 63 references that are in the map. First posted on arXiv 10 Mar 2022.

Builds on

48 of 63 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2016Wide Residual Networkscited by 3,057
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  8. 2016Adversarial examples in the physical worldcited by 1,762
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  11. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  12. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
36 more
  1. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  2. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 591
  3. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  4. 2020Overfitting in adversarially robust deep learningcited by 504
  5. 2019Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attackscited by 440
  6. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  7. 2017Mitigating Adversarial Effects Through Randomizationcited by 382
  8. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  9. 2019Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attackscited by 352
  10. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  11. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
  12. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  13. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 242
  14. 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
  15. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
  16. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  17. 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 181
  18. 2018Stochastic Activation Pruning for Robust Adversarial Defensecited by 171
  19. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
  20. 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
  21. 2020Boosting Adversarial Training with Hypersphere Embeddingcited by 97
  22. 2019Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Trainingcited by 95
  23. 2020Learnable Boundary Guided Adversarial Trainingcited by 93
  24. 2020Patch-wise Attack for Fooling Deep Neural Networkcited by 78
  25. 2019Controlling Neural Level Setscited by 76
  26. 2019Towards Large yet Imperceptible Adversarial Image Perturbations with Perceptual Color Distancecited by 72
  27. 2017Adversarial and Clean Data Are Not Twinscited by 52
  28. 2019An Alternative Surrogate Loss for PGD-based Adversarial Testingcited by 45
  29. 2022Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domainscited by 42
  30. 2020A Tunable Robust Pruning Framework Through Dynamic Network Rewiring of DNNscited by 37
  31. 2020Composite Adversarial Attackscited by 29
  32. 2021LAFEAT: Piercing Through Adversarial Defenses with Latent Featurescited by 24
  33. 2021Feature Space Targeted Attacks by Statistic Alignmentcited by 21
  34. 2020Patch-wise++ Perturbation for Adversarial Targeted Attackscited by 19
  35. 2020Manifold Regularization for Locally Stable Deep Neural Networkscited by 8
  36. 2022Practical No-box Adversarial Attacks with Training-free Hybrid Image Transformationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 5; 2024: 18; 2025: 7; 2026: 3.

44 papers.

Sort:

By area:

  1. 2022Frequency Domain Model Augmentation for Adversarial Attackcited by 128
  2. 2022Natural Color Fool: Towards Boosting Black-box Unrestricted Attackscited by 38
  3. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  4. 2024Exploring the Robustness of In-Context Learning with Noisy Labelscited by 13
  5. 2022Defense Against Adversarial Attacks on Audio DeepFake Detectioncited by 12
  6. 2024DAFA: Distance-Aware Fair Adversarial Trainingcited by 11
  7. 2023Benchmarking Robustness to Adversarial Image Obfuscationscited by 8
  8. 2024DD-RobustBench: An Adversarial Robustness Benchmark for Dataset Distillationcited by 8
  9. 2025PatchGuard: Adversarially Robust Anomaly Detection and Localization through Vision Transformers and Pseudo Anomaliescited by 8
  10. 2022Improving the Transferability of Adversarial Attacks on Face Recognition with Beneficial Perturbation Feature Augmentationcited by 7
  11. 2022MORA: Improving Ensemble Robustness Evaluation with Model-Reweighing Attackcited by 7
  12. 2025RemedyGS: Defend 3D Gaussian Splatting against Computation Cost Attackscited by 7
32 more
  1. 2024Adversarially Robust Out-of-Distribution Detection Using Lyapunov-Stabilized Embeddingscited by 6
  2. 2024Adversarial Training: A Surveycited by 6
  3. 2023A Comprehensive Study on the Robustness of Image Classification and Object Detection in Remote Sensing: Surveying and Benchmarkingcited by 5
  4. 2024Towards Adversarial Robustness via Debiased High-Confidence Logit Alignmentcited by 5
  5. 2024Towards Robust Vision Transformer via Masked Adaptive Ensemblecited by 4
  6. 2022Exposing Fine-Grained Adversarial Vulnerability of Face Anti-Spoofing Modelscited by 3
  7. 2024Enhancing the "Immunity" of Mixture-of-Experts Networks for Adversarial Defensecited by 3
  8. 2024Mitigating Low-Frequency Bias: Feature Recalibration and Frequency Attention Regularization for Adversarial Robustnesscited by 3
  9. 2024Towards Million-Scale Adversarial Robustness Evaluation With Stronger Individual Attackscited by 3
  10. 2022A Closer Look at Robustness to L-infinity and Spatial Perturbations and their Compositioncited by 2
  11. 2022Alternating Objectives Generates Stronger PGD-Based Adversarial Attackscited by 2
  12. 2023Evaluating Adversarial Robustness on Document Image Classificationcited by 2
  13. 2023Evaluation of Parameter-based Attacks against Embedded Neural Networks with Laser Injectioncited by 2
  14. 2024Improving Adversarial Robustness via Decoupled Visual Representation Maskingcited by 2
  15. 2025Are Time-Series Foundation Models Deployment-Ready? A Systematic Study of Adversarial Robustness Across Domainscited by 2
  16. 2025FrameShield: Adversarially Robust Video Anomaly Detectioncited by 2
  17. 2024SoK: Analyzing Adversarial Examples: A Framework to Study Adversary Knowledgecited by 1
  18. 2025Generating Adversarial Point Clouds Using Diffusion Modelcited by 1
  19. 2025Calibrating Uncertainty for Zero-Shot Adversarial CLIPcited by 1
  20. 2026MoCo-EA: Exploiting Adversarial Mode Connectivity for Efficient Evolutionary Attackscited by 1
  21. 2022CE-based white-box adversarial attacks will not work using super-fittingcited by 0
  22. 2022Rethinking Classifier and Adversarial Attackcited by 0
  23. 2022A Multi-objective Memetic Algorithm for Auto Adversarial Attack Optimization Designcited by 0
  24. 2023Multi-objective Evolutionary Search of Variable-length Composite Semantic Perturbationscited by 0
  25. 2024Redesigning Traffic Signs to Mitigate Machine-Learning Patch Attackscited by 0
  26. 2024Reinforced Compressive Neural Architecture Search for Versatile Adversarial Robustnesscited by 0
  27. 2024DataFreeShield: Defending Adversarial Attacks without Training Datacited by 0
  28. 2024Enhancing Output Diversity Improves Conjugate Gradient-based Adversarial Attackscited by 0
  29. 2024Keep on Swimming: Real Attackers Only Need Partial Knowledge of a Multi-Model Systemcited by 0
  30. 2025Tight Robustness Certificates and Wasserstein Distributional Attacks for Deep Neural Networkscited by 0
  31. 2026RPP: A Certified Poisoned-Sample Detection Framework for Backdoor Attacks under Dataset Imbalancecited by 0
  32. 2026Dummy-Aware Weighted Attack (DAWA): Breaking the Safe Sink in Dummy Class Defensescited by 0