mlsanityData to 14 Oct 2026

Exploring Adversarial Attacks and Defenses in Vision Transformers trained with DINO

arXiv 2206.06761 ↗First posted 14 Jun 2022cs.CV cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

This work conducts the first analysis on the robustness against adversarial attacks on self-supervised Vision Transformers trained using DINO. First, we evaluate whether features learned through self-supervision are more robust to adversarial attacks than those emerging from supervised learning. Then, we present properties arising for attacks in the latent space. Finally, we evaluate whether three well-known defense strategies can increase adversarial robustness in downstream tasks by only fine-tuning the classification head to provide robustness even in view of limited compute resources. These defense strategies are: Adversarial Training, Ensemble Adversarial Training and Ensemble of Specialized Networks.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 34 of its 41 references that are in the map. First posted on arXiv 14 Jun 2022.

Builds on

34 of 41 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  10. 2013Intriguing properties of neural networkscited by 5,155
  11. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  12. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
22 more
  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  2. 2016Practical Black-Box Attacks against Machine Learningcited by 903
  3. 2018Robustness May Be at Odds with Accuracycited by 880
  4. 2017Ensemble Adversarial Training: Attacks and Defensescited by 840
  5. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 666
  6. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  7. 2017Countering Adversarial Images using Input Transformationscited by 520
  8. 2017Detecting Adversarial Samples from Artifactscited by 245
  9. 2016A study of the effect of JPG compression on adversarial imagescited by 197
  10. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  11. 2020Torchattacks: A PyTorch Repository for Adversarial Attackscited by 158
  12. 2021On the Adversarial Robustness of Vision Transformerscited by 106
  13. 2020Parametric UMAP embeddings for representation and semi-supervised learningcited by 104
  14. 2017Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compressioncited by 87
  15. 2019The Odds are Odd: A Statistical Test for Detecting Adversarial Examplescited by 75
  16. 2017Robust Physical-World Attacks on Deep Learning Modelscited by 72
  17. 2021On Improving Adversarial Transferability of Vision Transformerscited by 70
  18. 2019ME-Net: Towards Effective Adversarial Robustness with Matrix Estimationcited by 65
  19. 2021Reveal of Vision Transformers Robustness against Adversarial Attackscited by 43
  20. 2019When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signaturescited by 40
  21. 2022"That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attackscited by 24
  22. 2017Robustness to Adversarial Examples through an Ensemble of Specialistscited by 21

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Towards Robust Semantic Segmentation against Patch-based Attack via Attention Refinementcited by 2
  2. 2025Robustness Tokens: Towards Adversarial Robustness of Transformerscited by 2
  3. 2024Liveness Detection in Computer Vision: Transformer-based Self-Supervised Learning for Face Anti-Spoofingcited by 1
  4. 2025Task-Agnostic Attacks Against Vision Foundation Modelscited by 1