mlsanityData to 7 Oct 2026

Do Adversarially Robust ImageNet Models Transfer Better?

arXiv 2007.08489 ↗First posted 16 Jul 2020cs.CV cs.LG stat.MLcited by 315

Not on the map. Why?

Abstract (arXiv)

Transfer learning is a widely-used paradigm in deep learning, where models pre-trained on standard datasets can be efficiently adapted to downstream tasks. Typically, better pre-trained models yield better transfer results, suggesting that initial accuracy is a key aspect of transfer learning performance. In this work, we identify another such aspect: we find that adversarially robust models, while less accurate, often perform better than their standard-trained counterparts when used for transfer learning. Specifically, we focus on adversarially robust ImageNet classifiers, and show that they yield improved accuracy on a standard suite of downstream classification tasks. Further analysis uncovers more differences between robust and standard models in the context of transfer learning. Our results are consistent with (and in fact, add to) recent hypotheses stating that robustness leads to improved feature representations. Our code and models are available at https://github.com/Microsoft/robust-models-transfer .

Key facts. Cited by 315 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 55; 2022: 86; 2023: 78; 2024: 42; 2025: 39; 2026: 15.

315 papers.

Sort:

By area:

  1. 2022Diffusion Models for Adversarial Purificationcited by 469
  2. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  3. 2021Partial success in closing the gap between human and machine visioncited by 185
  4. 2022On the Effectiveness of Parameter-Efficient Fine-Tuningcited by 140
  5. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 128
  6. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
  7. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  8. 2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Modelscited by 117
  9. 2022Improving Adversarial Transferability via Neuron Attribution-Based Attackscited by 92
  10. 2023Rethinking Model Ensemble in Transfer-based Adversarial Attackscited by 88
  11. 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
  12. 2021Leveraging Sparse Linear Layers for Debuggable Deep Networkscited by 82
38 more
  1. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  2. 2023Robust Classification via a Single Diffusion Modelcited by 79
  3. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
  4. 2022Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbationcited by 67
  5. 2021Improved OOD Generalization via Adversarial Training and Pre-trainingcited by 66
  6. 2023Diffusion-Based Adversarial Sample Generation for Improved Stealthiness and Controllabilitycited by 66
  7. 2021On Generating Transferable Targeted Perturbationscited by 65
  8. 2023Robust Evaluation of Diffusion-Based Adversarial Purificationcited by 65
  9. 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 65
  10. 2022A Light Recipe to Train Robust Vision Transformerscited by 64
  11. 2023Text-To-Concept (and Back) via Cross-Model Alignmentcited by 63
  12. 2022Harmonizing the object recognition strategies of deep neural networks with humanscited by 59
  13. 2021Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Trainingcited by 58
  14. 2022Transferability in Deep Learning: A Surveycited by 57
  15. 2024Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?cited by 56
  16. 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 55
  17. 2021Do Input Gradients Highlight Discriminative Features?cited by 52
  18. 2021How Well Do Sparse Imagenet Models Transfer?cited by 50
  19. 2022ViewFool: Evaluating the Robustness of Visual Recognition to Adversarial Viewpointscited by 48
  20. 2022Boosting Out-of-distribution Detection with Typical Featurescited by 48
  21. 2023Robust Principles: Architectural Design Principles for Adversarially Robust CNNscited by 46
  22. 2021Improved Regularization and Robustness for Fine-tuning in Neural Networkscited by 45
  23. 2021Frustratingly Easy Transferability Estimationcited by 44
  24. 2022Practical Evaluation of Adversarial Robustness via Adaptive Auto Attackcited by 44
  25. 2021How Fine-Tuning Allows for Effective Meta-Learningcited by 41
  26. 2021Adversarial Training Helps Transfer Learning via Better Representationscited by 36
  27. 2022Towards Understanding and Boosting Adversarial Transferability from a Distribution Perspectivecited by 36
  28. 2021Trivial or impossible -- dichotomous data difficulty masks model differences (on ImageNet and beyond)cited by 35
  29. 2023Does progress on ImageNet transfer to real-world datasets?cited by 35
  30. 2021A Little Robustness Goes a Long Way: Leveraging Robust Features for Targeted Transfer Attackscited by 33
  31. 2022DISCO: Adversarial Defense with Local Implicit Functionscited by 33
  32. 2022A Data-Based Perspective on Transfer Learningcited by 32
  33. 2023Don't trust your eyes: on the (un)reliability of feature visualizationscited by 32
  34. 2023Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferabilitycited by 32
  35. 2024Understanding and Improving Training-free Loss-based Diffusion Guidancecited by 31
  36. 2023ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editingcited by 30
  37. 2021On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learningcited by 29
  38. 2022FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfittingcited by 28

50 most cited of 315 All 315 (JSON)