mlsanityData to 7 Oct 2026

Reliable Evaluation of Adversarial Transferability

arXiv 2306.08565 ↗First posted 14 Jun 2023cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Adversarial examples (AEs) with small adversarial perturbations can mislead deep neural networks (DNNs) into wrong predictions. The AEs created on one DNN can also fool another DNN. Over the last few years, the transferability of AEs has garnered significant attention as it is a crucial property for facilitating black-box attacks. Many approaches have been proposed to improve adversarial transferability. However, they are mainly verified across different convolutional neural network (CNN) architectures, which is not a reliable evaluation since all CNNs share some similar architectural biases. In this work, we re-evaluate 12 representative transferability-enhancing attack methods where we test on 18 popular models from 4 types of neural networks. Our reevaluation revealed that the adversarial transferability is often overestimated, and there is no single AE that can be transferred to all popular models. The transferability rank of previous attacking methods changes when under our comprehensive evaluation. Based on our analysis, we propose a reliable benchmark including three evaluation protocols. Adversarial transferability on our new benchmark is extremely low, which further confirms the overestimation of adversarial transferability. We release our benchmark at https://adv-trans-eval.github.io to facilitate future research, which includes code, model checkpoints, and evaluation protocols.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 62 of its 80 references that are in the map. First posted on arXiv 14 Jun 2023.

Builds on

62 of 80 references.

References · 62

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  8. 2013Intriguing properties of neural networkscited by 5,114
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  10. 2014Going Deeper with Convolutionscited by 4,231
  11. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  12. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 591
38 more
  1. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
  2. 2019Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attackscited by 440
  3. 2021Dynamic Neural Networks: A Surveycited by 405
  4. 2019Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attackscited by 352
  5. 2018Deep Learning in Spiking Neural Networkscited by 258
  6. 2021Enhancing the Transferability of Adversarial Attacks through Variance Tuningcited by 242
  7. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  8. 2020Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNetscited by 165
  9. 2021Admix: Enhancing the Transferability of Adversarial Attackscited by 155
  10. 2019Enhancing Adversarial Example Transferability with an Intermediate Level Attackcited by 139
  11. 2021Feature Importance-aware Transferable Adversarial Attackscited by 125
  12. 2021A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibrationcited by 111
  13. 2020Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classificationcited by 99
  14. 2022Improving Adversarial Transferability via Neuron Attribution-Based Attackscited by 92
  15. 2020On Success and Simplicity: A Second Look at Transferable Targeted Attackscited by 89
  16. 2021Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferabilitycited by 84
  17. 2021Improving Transferability of Adversarial Patches on Face Recognition with Generative Modelscited by 80
  18. 2021Towards Transferable Adversarial Attacks on Vision Transformerscited by 80
  19. 2020Backpropagating Linearly Improves Transferability of Adversarial Examplescited by 79
  20. 2020A Unified Approach to Interpreting and Boosting Adversarial Transferabilitycited by 71
  21. 2021On Improving Adversarial Transferability of Vision Transformerscited by 70
  22. 2018Learning Transferable Adversarial Examples via Ghost Networkscited by 69
  23. 2022Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbationcited by 67
  24. 2020Perturbing Across the Feature Hierarchy to Improve Standard and Strict Blackbox Attack Transferabilitycited by 57
  25. 2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
  26. 2019FDA: Feature Disruptive Attackcited by 55
  27. 2020Transferable Perturbations of Deep Feature Distributionscited by 51
  28. 2022Improving the Transferability of Targeted Adversarial Examples through Object-Based Diverse Inputcited by 49
  29. 2021Are Vision Transformers Robust to Patch Perturbations?cited by 39
  30. 2022LGV: Boosting Adversarial Example Transferability from Large Geometric Vicinitycited by 39
  31. 2021Improving the Transferability of Adversarial Examples with Resized-Diverse-Inputs, Diversity-Ensemble and Region Fittingcited by 37
  32. 2020Batch Normalization Increases Adversarial Vulnerability and Decreases Adversarial Transferability: A Non-Robust Feature Perspectivecited by 35
  33. 2022Transferable Adversarial Attack based on Integrated Gradientscited by 35
  34. 2022An Impartial Take to the CNN vs Transformer Robustness Contestcited by 34
  35. 2023Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examplescited by 32
  36. 2022Towards Efficient Adversarial Training on Vision Transformerscited by 29
  37. 2022Investigating Top-k White-Box and Transferable Black-box Attackcited by 26
  38. 2019Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defensescited by 22

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 4; 2025: 1; 2026: 2.

12 papers.

  1. 2023A Survey on Transferability of Adversarial Examples across Deep Neural Networkscited by 43
  2. 2024An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Modelscited by 40
  3. 2023SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentationcited by 39
  4. 2024Semantic-Aligned Adversarial Evolution Triangle for High-Transferability Vision-Language Attackcited by 18
  5. 2023Improving Adversarial Transferability via Model Alignmentcited by 10
  6. 2023TranSegPGD: Improving Transferability of Adversarial Examples on Semantic Segmentationcited by 10
  7. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8
  8. 2024Distilling Adversarial Robustness Using Heterogeneous Teacherscited by 3
  9. 2023Exploring Non-additive Randomness on ViT against Query-Based Black-Box Attackscited by 1
  10. 2026Deepfake detectors are DUMB: A benchmark to assess adversarial training robustness under transferability constraintscited by 1
  11. 2025Optimizing the Adversarial Perturbation with a Momentum-based Adaptive Matrixcited by 0
  12. 2026RAPID: A Real-Time Defense Against Unauthorized Model Distillation for Text-to-Image Servicescited by 0