mlsanityData to 14 Oct 2026

Defending Backdoor Attacks on Vision Transformer via Patch Processing

arXiv 2206.12381 ↗First posted 24 Jun 2022cs.CVcited by 20

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Vision Transformers (ViTs) have a radically different architecture with significantly less inductive bias than Convolutional Neural Networks. Along with the improvement in performance, security and robustness of ViTs are also of great importance to study. In contrast to many recent works that exploit the robustness of ViTs against adversarial examples, this paper investigates a representative causative attack, i.e., backdoor. We first examine the vulnerability of ViTs against various backdoor attacks and find that ViTs are also quite vulnerable to existing attacks. However, we observe that the clean-data accuracy and backdoor attack success rate of ViTs respond distinctively to patch transformations before the positional encoding. Then, based on this finding, we propose an effective method for ViTs to defend both patch-based and blending-based trigger backdoor attacks via patch processing. The performances are evaluated on several benchmark datasets, including CIFAR10, GTSRB, and TinyImageNet, which show the proposed novel defense is very successful in mitigating backdoor attacks for ViTs. To the best of our knowledge, this paper presents the first defensive strategy that utilizes a unique characteristic of ViTs against backdoor attacks. The paper will appear in the Proceedings of the AAAI'23 Conference. This work was initially submitted in November 2021 to CVPR'22, then it was re-submitted to ECCV'22. The paper was made public in June 2022. The authors sincerely thank all the referees from the Program Committees of CVPR'22, ECCV'22, and AAAI'23.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 45 of its 66 references that are in the map. First posted on arXiv 24 Jun 2022.

Builds on

45 of 66 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021DBIA: Data-free Backdoor Injection Attack against Transformer Networks22 Nov

Other references · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  8. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  9. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  10. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  11. 2021Transformers in Vision: A Surveycited by 1,013
  12. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
32 more
  1. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 580
  2. 2018Spectral Signatures in Backdoor Attackscited by 484
  3. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 460
  4. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 459
  5. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 456
  6. 2021Intriguing Properties of Vision Transformerscited by 446
  7. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 403
  8. 2019Hidden Trigger Backdoor Attackscited by 371
  9. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 349
  10. 2021Understanding Robustness of Transformers for Image Classificationcited by 249
  11. 2019A new Backdoor Attack in CNNs by training set corruption without label poisoningcited by 242
  12. 2020Blind Backdoors in Deep Learning Modelscited by 182
  13. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  14. 2017Neural Trojanscited by 153
  15. 2021Towards Robust Vision Transformercited by 137
  16. 2019Defending Neural Backdoors via Generative Distribution Modelingcited by 84
  17. 2020Rethinking the Trigger of Backdoor Attackcited by 77
  18. 2020DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentationcited by 77
  19. 2021On Improving Adversarial Transferability of Vision Transformerscited by 70
  20. 2018Backdooring Convolutional Neural Networks via Targeted Weight Perturbationscited by 69
  21. 2020Deep Learning COVID-19 Features on CXR using Limited Training Data Setscited by 68
  22. 2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
  23. 2019Model Agnostic Defence against Backdoor Attacks in Machine Learningcited by 52
  24. 2021MVT: Multi-view Vision Transformer for 3D Object Recognitioncited by 34
  25. 2019Deep Learning for Physical-Layer 5G Wireless Techniques: Opportunities, Challenges and Solutionscited by 29
  26. 2020Defending against Backdoor Attack on Deep Neural Networkscited by 22
  27. 2022BOAT: Bilateral Local Attention Vision Transformercited by 12
  28. 2021Investigating the Vision Transformer Model for Image Retrieval Taskscited by 11
  29. 2021Adversarial Token Attacks on Vision Transformerscited by 11
  30. 2022One Loss for Quantization: Deep Hashing with Discrete Wasserstein Distributional Matchingcited by 9
  31. 2022Integrity Authentication in Tree Modelscited by 4
  32. 2020Image Hashing by Minimizing Discrete Component-wise Wasserstein Distancecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 3; 2024: 8; 2025: 1; 2026: 5.

20 papers.

Sort:

By area:

  1. 2022Marksman Backdoor: Backdoor Attacks with Arbitrary Target Classcited by 33
  2. 2022TrojViT: Trojan Insertion in Vision Transformerscited by 31
  3. 2024DisDet: Exploring Detectability of Backdoor Attack on Diffusion Modelscited by 17
  4. 2023TextGuard: Provable Defense against Backdoor Attacks on Text Classificationcited by 16
  5. 2023Boosting Backdoor Attack with A Learnable Poisoning Sample Selection Strategycited by 5
  6. 2024Not All Prompts Are Secure: A Switchable Backdoor Attack Against Pre-trained Vision Transformerscited by 5
  7. 2024Perturb and Recover: Fine-tuning for Effective Backdoor Removal from CLIPcited by 4
  8. 2024Backdoor Attack Against Vision Transformers via Attention Gradient-Based Image Erosioncited by 2
  9. 2026DF-LoGiT: Data-Free Logic-Gated Backdoor Attacks in Vision Transformerscited by 2
  10. 2022Degenerate Swin to Win: Plain Window-based Transformer without Sophisticated Operationscited by 1
  11. 2023Exploring Non-additive Randomness on ViT against Query-Based Black-Box Attackscited by 1
  12. 2024TokenMark: A Modality-Agnostic Watermark for Pre-trained Transformerscited by 1
8 more
  1. 2024BadVim: Unveiling Backdoor Threats in Visual State Space Modelcited by 1
  2. 2024Using Interleaved Ensemble Unlearning to Keep Backdoors at Bay for Finetuning Vision Transformerscited by 1
  3. 2026Backdoor Directions in Vision Transformerscited by 1
  4. 2026BackdoorIDS: Zero-shot Backdoor Detection for Pretrained Vision Encodercited by 1
  5. 2024An Effective and Resilient Backdoor Attack Framework against Deep Neural Networks and Vision Transformerscited by 0
  6. 2025Backdoor Attacks on Open Vocabulary Object Detectors via Multi-Modal Prompt Tuningcited by 0
  7. 2026IU: Imperceptible Universal Backdoor Attackcited by 0
  8. 2026PASTA: A Patch-Agnostic Twofold-Stealthy Backdoor Attack on Vision Transformerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Backdoor Attacks on Vision Transformers
  2. 2024One Pixel is All I Need
  3. 2026A Survey of Adversarial Efficiency Degradation for Vision Transformer by Exploiting Input-adaptive Optimization
  4. 2026MOAT: Model-Agnostic Randomized Transformations for preventing Efficiency Degradation Attacks on ViTs
  5. 2025Pruning and Malicious Injection: A Retraining-Free Backdoor Attack on Transformer Models
  6. 2025Harnessing the Computation Redundancy in ViTs to Boost Adversarial Transferability
  7. 2024Context is the Key: Backdoor Attacks for In-Context Learning with Vision Transformers
  8. 2025Breaking the Illusion of Security via Interpretation: Interpretable Vision Transformer Systems under Attack