mlsanityData to 14 Oct 2026

Increasing Confidence in Adversarial Robustness Evaluations

arXiv 2206.13991 ↗First posted 28 Jun 2022cs.LG cs.CR cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Hundreds of defenses have been proposed to make deep neural networks robust against minimal (adversarial) input perturbations. However, only a handful of these defenses held up their claims because correctly evaluating robustness is extremely challenging: Weak attacks often fail to find adversarial examples even if they unknowingly exist, thereby making a vulnerable network look robust. In this paper, we propose a test to identify weak attacks, and thus weak defense evaluations. Our test slightly modifies a neural network to guarantee the existence of an adversarial example for every sample. Consequentially, any correct attack must succeed in breaking this modified network. For eleven out of thirteen previously-published defenses, the original evaluation of the defense fails our test, while stronger attacks that break these defenses pass it. We hope that attack unit tests - such as ours - will be a major component in future robustness evaluations and increase confidence in an empirical field that is currently riddled with skepticism.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 38 of its 55 references that are in the map. First posted on arXiv 28 Jun 2022.

Builds on

38 of 55 references.

References · 38

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  2. 2013Intriguing properties of neural networkscited by 5,155
  3. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  4. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  5. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  6. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  7. 2017Evasion Attacks against Machine Learning at Test Timecited by 689
  8. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  9. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
  10. 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
  11. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  12. 2017Countering Adversarial Images using Input Transformationscited by 520
26 more
  1. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 501
  2. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 468
  3. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
  4. 2019On Evaluating Adversarial Robustnesscited by 447
  5. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
  6. 2017PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examplescited by 244
  7. 2021Improving Robustness using Generated Datacited by 234
  8. 2017On Detecting Adversarial Perturbationscited by 232
  9. 2021Data Augmentation Can Improve Robustnesscited by 194
  10. 2018Towards the first adversarially robust neural network model on MNISTcited by 104
  11. 2019Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Trainingcited by 95
  12. 2018GenAttack: Practical Black-box Attacks with Gradient-Free Optimizationcited by 82
  13. 2019The Odds are Odd: A Statistical Test for Detecting Adversarial Examplescited by 75
  14. 2015Adversarial Manipulation of Deep Representationscited by 56
  15. 2019Adversarial Defense by Restricting the Hidden Space of Deep Neural Networkscited by 56
  16. 2019Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attackscited by 54
  17. 2021Score-Based Generative Classifierscited by 53
  18. 2019Enhancing Adversarial Defense by k-Winners-Take-Allcited by 51
  19. 2019ML-LOO: Detecting Adversarial Examples with Feature Attributioncited by 48
  20. 2019Accurate, reliable and fast robustness evaluationcited by 43
  21. 2019Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networkscited by 40
  22. 2021Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descentcited by 35
  23. 2020EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attackscited by 28
  24. 2021Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examplescited by 23
  25. 2019Defending Against Adversarial Examples with K-Nearest Neighborcited by 5
  26. 2021Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided Curriculum Learning Approachcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 5; 2024: 5.

12 papers.

  1. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  2. 2023Are aligned neural networks adversarially aligned?cited by 315
  3. 2023Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulationcited by 61
  4. 2022A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunitiescited by 9
  5. 2024Calibration Attacks: A Comprehensive Study of Adversarial Attacks on Model Confidencecited by 7
  6. 2022A Closer Look at Evaluating the Bit-Flip Attack Against Deep Neural Networkscited by 5
  7. 2024SongBsAb: A Dual Prevention Approach against Singing Voice Conversion based Illegal Song Coverscited by 5
  8. 2024SoundnessBench: A Soundness Benchmark for Neural Network Verifierscited by 5
  9. 2024On the Effectiveness of Adversarial Training on Malware Classifierscited by 1
  10. 2023A Minimax Approach Against Multi-Armed Adversarial Attacks Detectioncited by 0
  11. 2023Two Heads are Actually Better than One: Towards Better Adversarial Robustness via Transduction and Rejectioncited by 0
  12. 2024Optimal Zero-Shot Detector for Multi-Armed Attackscited by 0