SoK: Analyzing Adversarial Examples: A Framework to Study Adversary Knowledge
Not on the map. Why?
Abstract (arXiv)
Adversarial examples are malicious inputs to machine learning models that trigger a misclassification. This type of attack has been studied for close to a decade, and we find that there is a lack of study and formalization of adversary knowledge when mounting attacks. This has yielded a complex space of attack research with hard-to-compare threat models and attacks. We focus on the image classification domain and provide a theoretical framework to study adversary knowledge inspired by work in order theory. We present an adversarial example game, inspired by cryptographic games, to standardize attacks. We survey recent attacks in the image classification domain and classify their adversary's knowledge in our framework. From this systematization, we compile results that both confirm existing beliefs about adversary knowledge, such as the potency of information about the attacked model as well as allow us to derive new conclusions on the difficulty associated with the white-box and transferable threat models, for example, that transferable attacks might not be as difficult as previously thought.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 74 of its 116 references that are in the map. First posted on arXiv 22 Feb 2024.
Builds on
74 of 116 references.
References · 74
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2016Densely Connected Convolutional Networkscited by 6,691
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2013Intriguing properties of neural networkscited by 5,114
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
- 2016Wide Residual Networkscited by 3,057
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
- 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
- 2020Fast is better than free: Revisiting adversarial trainingcited by 723
38 more
- 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
- 2020Overfitting in adversarially robust deep learningcited by 504
- 2022Diffusion Models for Adversarial Purificationcited by 469
- 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
- 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
- 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
- 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
- 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
- 2023Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examplescited by 163
- 2021Admix: Enhancing the Transferability of Adversarial Attackscited by 155
- 2022Frequency Domain Model Augmentation for Adversarial Attackcited by 128
- 2022Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenoncited by 101
- 2019Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Trainingcited by 95
- 2020Learnable Boundary Guided Adversarial Trainingcited by 93
- 2023Diffusion Models for Imperceptible and Transferable Adversarial Attackcited by 93
- 2022Frequency-driven Imperceptible Adversarial Attack on Semantic Similaritycited by 76
- 2022Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbationcited by 67
- 2023Robust Evaluation of Diffusion-Based Adversarial Purificationcited by 65
- 2018Motivating the Rules of the Game for Adversarial Example Researchcited by 57
- 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 55
- 2021Adversarial Example Detection for DNN Models: A Review and Experimental Comparisoncited by 51
- 2023Boosting Adversarial Transferability by Achieving Flat Local Maximacited by 48
- 2021Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Themcited by 46
- 2023AdvDiff: Generating Unrestricted Adversarial Examples using Diffusion Modelscited by 46
- 2020The Intriguing Relation Between Counterfactual Explanations and Adversarial Examplescited by 44
- 2022Practical Evaluation of Adversarial Robustness via Adaptive Auto Attackcited by 44
- 2022A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologiescited by 44
- 2022Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domainscited by 42
- 2023CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via Adversarial Latent Searchcited by 42
- 2023An Adaptive Model Ensemble Adversarial Attack for Boosting Adversarial Transferabilitycited by 42
- 2022LGV: Boosting Adversarial Example Transferability from Large Geometric Vicinitycited by 39
- 2020Adversarial Example Gamescited by 34
- 2023Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examplescited by 32
- 2022Enhancing the Self-Universality for Transferable Targeted Attackscited by 30
- 2021A Review of Adversarial Attack and Defense for Classification Methodscited by 29
- 2021Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examplescited by 23
- 2021Unrestricted Adversarial Attacks on ImageNet Competitioncited by 23
- 2020AI-GAN: Attack-Inspired Generation of Adversarial Examplescited by 22
50 most cited of 74 All 74 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026SoK: Colluding Adversaries in Machine Learning Pipelinescited by 0
- 2026SoK: Colluding Adversaries in Machine Learning Pipelines8 Jun 2026