mlsanityData to 14 Oct 2026

Does Robustness on ImageNet Transfer to Downstream Tasks?

arXiv 2204.03934 ↗First posted 8 Apr 2022cs.CV cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

As clean ImageNet accuracy nears its ceiling, the research community is increasingly more concerned about robust accuracy under distributional shifts. While a variety of methods have been proposed to robustify neural networks, these techniques often target models trained on ImageNet classification. At the same time, it is a common practice to use ImageNet pretrained backbones for downstream tasks such as object detection, semantic segmentation, and image classification from different domains. This raises a question: Can these robust image classifiers transfer robustness to downstream tasks? For object detection and semantic segmentation, we find that a vanilla Swin Transformer, a variant of Vision Transformer tailored for dense prediction tasks, transfers robustness better than Convolutional Neural Networks that are trained to be robust to the corrupted version of ImageNet. For CIFAR10 classification, we find that models that are robustified for ImageNet do not retain robustness when fully fine-tuned. These findings suggest that current robustification techniques tend to emphasize ImageNet evaluations. Moreover, network architecture is a strong source of robustness when we consider transfer learning.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 33 of its 41 references that are in the map. First posted on arXiv 8 Apr 2022.

Builds on

33 of 41 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2017Mask R-CNNcited by 7,277
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  7. 2013Intriguing properties of neural networkscited by 5,155
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  9. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  10. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  11. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  12. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
21 more
  1. 2019Natural Adversarial Examplescited by 1,408
  2. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  3. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
  4. 2018Robustness May Be at Odds with Accuracycited by 880
  5. 2018Do Better ImageNet Models Transfer Better?cited by 565
  6. 2021Intriguing Properties of Vision Transformerscited by 446
  7. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 343
  8. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 336
  9. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 316
  10. 2021Understanding Robustness of Transformers for Image Classificationcited by 249
  11. 2016Speed/accuracy trade-offs for modern convolutional object detectorscited by 198
  12. 2019Benchmarking the Robustness of Semantic Segmentation Modelscited by 111
  13. 2019Interpreting Adversarially Trained Convolutional Neural Networkscited by 75
  14. 2019Adversarially robust transfer learningcited by 67
  15. 2019Do Image Classifiers Generalize Across Time?cited by 59
  16. 2019Towards Adversarially Robust Object Detectioncited by 55
  17. 2020Robust Object Detection under Occlusion with Context-Aware CompositionalNetscited by 54
  18. 2021Robust and Accurate Object Detection via Adversarial Learningcited by 42
  19. 2021PRIME: A few primitives can boost robustness to common corruptionscited by 40
  20. 2019What it Thinks is Important is Important: Robustness Transfers through Input Gradientscited by 34
  21. 2020Increasing the Robustness of Semantic Segmentation Models with Painting-by-Numberscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 10; 2024: 3; 2025: 2; 2026: 2.

20 papers.

Sort:

By area:

  1. 2022Towards Robust Referring Image Segmentationcited by 49
  2. 2022Robustifying Vision Transformer without Retraining from Scratch by Test-Time Class-Conditional Feature Alignmentcited by 27
  3. 2023Strong but simple: A Baseline for Domain Generalized Dense Perception by CLIP-based Transfer Learningcited by 20
  4. 2023ConvNet vs Transformer, Supervised vs CLIP: Beyond ImageNet Accuracycited by 19
  5. 2023Benchmarking the Robustness of LiDAR Semantic Segmentation Modelscited by 18
  6. 2023TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalizationcited by 18
  7. 2024On Catastrophic Inheritance of Large Foundation Modelscited by 15
  8. 2024As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks?cited by 11
  9. 2023Initialization Matters for Adversarial Transfer Learningcited by 10
  10. 2022On Transfer of Adversarial Robustness from Pretraining to Downstream Taskscited by 6
  11. 2023Towards Efficient Task-Driven Model Reprogramming with Foundation Modelscited by 5
  12. 2023An Empirical Study of Pre-trained Model Selection for Out-of-Distribution Generalization and Calibrationcited by 5
8 more
  1. 2023Can Pre-trained Networks Detect Familiar Out-of-Distribution Data?cited by 4
  2. 2023Robust Tickets Can Transfer Better: Drawing More Transferable Subnetworks in Transfer Learningcited by 3
  3. 2025Examining the Impact of Optical Aberrations to Image Classification and Object Detection Modelscited by 2
  4. 2026Robust Onion: Peeling Open Vocab Object Detectors Under Noisecited by 1
  5. 2023A Theoretical Perspective on Subnetwork Contributions to Adversarial Robustnesscited by 0
  6. 2024D-Cube: Exploiting Hyper-Features of Diffusion Model for Robust Medical Classificationcited by 0
  7. 2025SNAP: A Benchmark for Testing the Effects of Capture Conditions on Fundamental Vision Taskscited by 0
  8. 2026Comparison of Image Processing Models in Quark Gluon Jet Classificationcited by 0