mlsanityData to 14 Oct 2026

When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture

arXiv 2210.07540 ↗First posted 14 Oct 2022cs.CV cs.LGcited by 64

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs) have recently achieved competitive performance in broad vision tasks. Unfortunately, on popular threat models, naturally trained ViTs are shown to provide no more adversarial robustness than convolutional neural networks (CNNs). Adversarial training is still required for ViTs to defend against such adversarial attacks. In this paper, we provide the first and comprehensive study on the adversarial training recipe of ViTs via extensive evaluation of various training techniques across benchmark datasets. We find that pre-training and SGD optimizer are necessary for ViTs' adversarial training. Further considering ViT as a new type of model architecture, we investigate its adversarial robustness from the perspective of its unique architectural components. We find, when randomly masking gradients from some attention blocks or masking perturbations on some patches during adversarial training, the adversarial robustness of ViTs can be remarkably improved, which may potentially open up a line of work to explore the architectural information inside the newly designed models like ViTs. Our code is available at https://github.com/mo666666/When-Adversarial-Training-Meets-Vision-Transformers.

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). Builds on 35 of its 54 references that are in the map. First posted on arXiv 14 Oct 2022.

Builds on

35 of 54 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  8. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  9. 2013Intriguing properties of neural networkscited by 5,155
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  11. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  12. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
23 more
  1. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  2. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  3. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  4. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  5. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  6. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  7. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
  8. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  9. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 661
  10. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
  11. 2020Overfitting in adversarially robust deep learningcited by 506
  12. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  13. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
  14. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 278
  15. 2020Bag of Tricks for Adversarial Trainingcited by 200
  16. 2021Are Transformers More Robust Than CNNs?cited by 187
  17. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  18. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  19. 2021On the Convergence and Robustness of Adversarial Trainingcited by 130
  20. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  21. 2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
  22. 2021Reveal of Vision Transformers Robustness against Adversarial Attackscited by 43
  23. 2022Self-Ensemble Adversarial Training for Improved Robustnesscited by 38

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 17; 2024: 20; 2025: 17; 2026: 10.

64 papers.

Sort:

By area:

  1. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  2. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuningcited by 67
  3. 2023CFA: Class-wise Calibrated Fair Adversarial Trainingcited by 53
  4. 2023Robust Principles: Architectural Design Principles for Adversarially Robust CNNscited by 46
  5. 2024Revisiting Adversarial Training at Scalecited by 33
  6. 2024TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoorscited by 25
  7. 2024On the Duality Between Sharpness-Aware Minimization and Adversarial Trainingcited by 24
  8. 2023Language-Driven Anchors for Zero-Shot Adversarial Robustnesscited by 23
  9. 2023Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspectivecited by 18
  10. 2025One Surrogate to Fool Them All: Universal, Transferable, and Targeted Adversarial Attacks with CLIPcited by 15
  11. 2023Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detectioncited by 14
  12. 2023Generalist: Decoupling Natural and Robust Generalizationcited by 14
38 more
  1. 2023Sharpness-Aware Minimization Alone can Improve Adversarial Robustnesscited by 14
  2. 2023On the Importance of Backbone to the Adversarial Robustness of Object Detectorscited by 12
  3. 2023MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustnesscited by 12
  4. 2025E2AT: Multimodal Jailbreak Defense via Dynamic Joint Optimization for Multimodal Large Language Modelscited by 12
  5. 2024FullLoRA: Efficiently Boosting the Robustness of Pretrained Vision Transformerscited by 9
  6. 2023Beyond Pretrained Features: Noisy Image Modeling Provides Adversarial Defensecited by 7
  7. 2024σ-zero: Gradient-based Optimization of ℓ0-norm Adversarial Examplescited by 7
  8. 2023RobArch: Designing Robust Architectures against Adversarial Attackscited by 6
  9. 2023AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentationcited by 6
  10. 2024PID: Prompt-Independent Data Protection Against Latent Diffusion Modelscited by 6
  11. 2024Adversarial Training: A Surveycited by 6
  12. 2024SpecFormer: Guarding Vision Transformer Robustness via Maximum Singular Value Penalizationcited by 5
  13. 2023Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filterscited by 4
  14. 2024Catastrophic Overfitting: A Potential Blessing in Disguisecited by 4
  15. 2024Watch the Watcher! Backdoor Attacks on Security-Enhancing Diffusion Modelscited by 4
  16. 2024Mitigating Low-Frequency Bias: Feature Recalibration and Frequency Attention Regularization for Adversarial Robustnesscited by 3
  17. 2025SAFER: Sharpness Aware layer-selective Finetuning for Enhanced Robustness in vision transformerscited by 3
  18. 2025Boosting Generative Adversarial Transferability with Self-supervised Vision Transformer Featurescited by 3
  19. 2023RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillationcited by 2
  20. 2024ViTGuard: Attention-aware Detection against Adversarial Examples for Vision Transformercited by 2
  21. 2024Hyper Adversarial Tuning for Boosting Adversarial Robustness of Pretrained Large Vision Modelscited by 2
  22. 2025Amnesia as a Catalyst for Enhancing Black Box Pixel Attacks in Image Classification and Object Detectioncited by 2
  23. 2025TESSER: Transfer-Enhancing Adversarial Attacks from Vision Transformers via Spectral and Semantic Regularizationcited by 2
  24. 2025TAIGen: Training-Free Adversarial Image Generation via Diffusion Modelscited by 2
  25. 2023Blacksmith: Fast Adversarial Training of Vision Transformers via a Mixture of Single-step and Multi-step Methodscited by 1
  26. 2024Semantic-Syntactic Discrepancy in Images (SSDI): Learning Meaning and Order of Features from Natural Imagescited by 1
  27. 2024S-E Pipeline: A Vision Transformer (ViT) based Resilient Classification Pipeline for Medical Imaging Against Adversarial Attackscited by 1
  28. 2024On the Adversarial Transferability of Generalized "Skip Connections"cited by 1
  29. 2025Defending Against Frequency-Based Attacks with Diffusion Modelscited by 1
  30. 2025ViT-EnsembleAttack: Augmenting Ensemble Models for Stronger Adversarial Transferability in Vision Transformerscited by 1
  31. 2025A Guide to Robust Generalization: The Impact of Architecture, Pre-training, and Optimization Strategycited by 1
  32. 2025Generalist++: A Meta-learning Framework for Mitigating Trade-off in Adversarial Trainingcited by 1
  33. 2025Parameter Interpolation Adversarial Training for Robust Image Classificationcited by 1
  34. 2026Solving adversarial examples requires solving exponential misalignmentcited by 1
  35. 2026Efficient Adversarial Training via Criticality-Aware Fine-Tuningcited by 1
  36. 2026Mitigating Error Amplification in Fast Adversarial Trainingcited by 1
  37. 2026Taming the Long Tail: Rebalancing Adversarial Training via Adaptive Perturbationcited by 1
  38. 2023STR-Cert: Robustness Certification for Deep Text Recognition on Deep Learning Pipelines and Vision Transformerscited by 0

50 most cited of 64 All 64 (JSON)