mlsanityData to 7 Oct 2026

Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularization

arXiv 2303.15754 ↗First posted 28 Mar 2023cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

Vision transformers (ViTs) have been successfully deployed in a variety of computer vision tasks, but they are still vulnerable to adversarial samples. Transfer-based attacks use a local model to generate adversarial samples and directly transfer them to attack a target black-box model. The high efficiency of transfer-based attacks makes it a severe security threat to ViT-based applications. Therefore, it is vital to design effective transfer-based attacks to identify the deficiencies of ViTs beforehand in security-sensitive scenarios. Existing efforts generally focus on regularizing the input gradients to stabilize the updated direction of adversarial samples. However, the variance of the back-propagated gradients in intermediate blocks of ViTs may still be large, which may make the generated adversarial samples focus on some model-specific features and get stuck in poor local optima. To overcome the shortcomings of existing approaches, we propose the Token Gradient Regularization (TGR) method. According to the structural characteristics of ViTs, TGR reduces the variance of the back-propagated gradient in each internal block of ViTs in a token-wise manner and utilizes the regularized gradient to generate adversarial samples. Extensive experiments on attacking both ViTs and CNNs confirm the superiority of our approach. Notably, compared to the state-of-the-art transfer-based attacks, our TGR offers a performance improvement of 8.8% on average.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 36 of its 45 references that are in the map. First posted on arXiv 28 Mar 2023.

Builds on

36 of 45 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  9. 2016Adversarial examples in the physical worldcited by 1,762
  10. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  11. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  12. 2021Going deeper with Image Transformerscited by 759
24 more
  1. 2021Transformer in Transformercited by 632
  2. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 591
  3. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  4. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  5. 2019Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attackscited by 440
  6. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  7. 2019Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attackscited by 352
  8. 2021Understanding Robustness of Transformers for Image Classificationcited by 250
  9. 2021Enhancing the Transferability of Adversarial Attacks through Variance Tuningcited by 242
  10. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  11. 2020Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNetscited by 165
  12. 2021Visformer: The Vision-friendly Transformercited by 137
  13. 2021On the Adversarial Robustness of Vision Transformerscited by 104
  14. 2022Improving Adversarial Transferability via Neuron Attribution-Based Attackscited by 92
  15. 2021Towards Transferable Adversarial Attacks on Vision Transformerscited by 80
  16. 2021On Improving Adversarial Transferability of Vision Transformerscited by 70
  17. 2021Boosting Adversarial Transferability through Enhanced Momentumcited by 53
  18. 2021Triangle Attack: A Query-efficient Decision-based Adversarial Attackcited by 48
  19. 2020Rethinking the Value of Transformer Componentscited by 24
  20. 2022AEON: A Method for Automatic Evaluation of NLP Test Casescited by 15
  21. 2022Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translationcited by 14
  22. 2022A2: Efficient Automated Attacker for Boosting Adversarial Trainingcited by 10
  23. 2021Surrogate Representation Learning with Isometric Mapping for Gray-box Graph Adversarial Attackscited by 4
  24. 2023Improving Adversarial Transferability with Scheduled Step Size and Dual Examplecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 14; 2025: 17; 2026: 4.

42 papers.

Sort:

By area:

  1. 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 65
  2. 2023SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentationcited by 39
  3. 2023On the Robustness of Latent Diffusion Modelscited by 21
  4. 2024Bag of Tricks to Boost Adversarial Transferabilitycited by 12
  5. 2024As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks?cited by 11
  6. 2024Large Language Models are Good Multi-lingual Learners : When LLMs Meet Cross-lingual Promptscited by 10
  7. 2025Towards Model Resistant to Transferable Adversarial Examples via Trigger Activationcited by 10
  8. 2025AdPO: Enhancing the Adversarial Robustness of Large Vision-Language Models with Preference Optimizationcited by 8
  9. 2025Improving Adversarial Transferability on Vision Transformers via Forward Propagation Refinementcited by 7
  10. 2024Adversarial Training: A Surveycited by 6
  11. 2025FOCUS: Fused Observation of Channels for Unveiling Spectracited by 6
  12. 2024Benchmarking Transferable Adversarial Attackscited by 5
30 more
  1. 2025Benchmarking Unified Face Attack Detection via Hierarchical Prompt Tuningcited by 5
  2. 2023Identifying Performance Issues in Cloud Service Systems Based on Relational-Temporal Featurescited by 4
  3. 2023Backpropagation Path Search On Adversarial Transferabilitycited by 4
  4. 2025Harnessing the Computation Redundancy in ViTs to Boost Adversarial Transferabilitycited by 4
  5. 2024Attacking Transformers with Feature Diversity Adversarial Perturbationcited by 3
  6. 2025Boosting Generative Adversarial Transferability with Self-supervised Vision Transformer Featurescited by 3
  7. 2023Vulnerability Analysis of Transformer-based Optical Character Recognition to Adversarial Attackscited by 2
  8. 2023QuantAttack: Exploiting Dynamic Quantization to Attack Vision Transformerscited by 2
  9. 2024DeSparsify: Adversarial Attack Against Token Sparsification Mechanisms in Vision Transformerscited by 2
  10. 2024Artificial Immune System of Secure Face Recognition Against Adversarial Attackscited by 2
  11. 2024ViTGuard: Attention-aware Detection against Adversarial Examples for Vision Transformercited by 2
  12. 2025Boosting the Local Invariance for Better Adversarial Transferabilitycited by 2
  13. 2025Attacking Attention of Foundation Models Disrupts Downstream Taskscited by 2
  14. 2024Semantic-Syntactic Discrepancy in Images (SSDI): Learning Meaning and Order of Features from Natural Imagescited by 1
  15. 2024On Inherent Adversarial Robustness of Active Vision Systemscited by 1
  16. 2024Downstream Transfer Attack: Adversarial Attacks on Downstream Models with Pre-trained Vision Transformerscited by 1
  17. 2024On the Adversarial Transferability of Generalized "Skip Connections"cited by 1
  18. 2024Evaluating the Adversarial Robustness of Detection Transformerscited by 1
  19. 2025Boosting Adversarial Transferability with Spatial Adversarial Alignmentcited by 1
  20. 2025TESSER: Transfer-Enhancing Adversarial Attacks from Vision Transformers via Spectral and Semantic Regularizationcited by 1
  21. 2025Boosting Adversarial Transferability via Commonality-Oriented Gradient Optimizationcited by 1
  22. 2026Grounding-Driven Attack: Improving Encoder-based Adversarial Transferability against Large Vision-Language Modelscited by 1
  23. 2026Frequency-Domain Regularized Adversarial Alignment for Transferable Attacks against Closed-Source MLLMscited by 1
  24. 2025Enabling Heterogeneous Adversarial Transferability via Feature Permutation Attackscited by 0
  25. 2025Breaking the Illusion of Security via Interpretation: Interpretable Vision Transformer Systems under Attackcited by 0
  26. 2025Vision Transformers: the threat of realistic adversarial patchescited by 0
  27. 2025SAFER-AiD: Saccade-Assisted Foveal-peripheral vision Enhanced Reconstruction for Adversarial Defensecited by 0
  28. 2025Modulo Video Recovery via Selective Spatiotemporal Vision Transformercited by 0
  29. 2026Devling into Adversarial Transferability on Image Classification: Review, Benchmark, and Evaluationcited by 0
  30. 2026Towards Universal Physical Adversarial Attacks via a Joint Multi-Objective and Multi-Model Optimization Frameworkcited by 0