mlsanityData to 7 Oct 2026

An Impartial Take to the CNN vs Transformer Robustness Contest

arXiv 2207.11347 ↗First posted 22 Jul 2022cs.CV cs.LGcited by 34

Not on the map. Why?

Abstract (arXiv)

Following the surge of popularity of Transformers in Computer Vision, several studies have attempted to determine whether they could be more robust to distribution shifts and provide better uncertainty estimates than Convolutional Neural Networks (CNNs). The almost unanimous conclusion is that they are, and it is often conjectured more or less explicitly that the reason of this supposed superiority is to be attributed to the self-attention mechanism. In this paper we perform extensive empirical analyses showing that recent state-of-the-art CNNs (particularly, ConvNeXt) can be as robust and reliable or even sometimes more than the current state-of-the-art Transformers. However, there is no clear winner. Therefore, although it is tempting to state the definitive superiority of one family of architectures over another, they seem to enjoy similar extraordinary performances on a variety of tasks while also suffering from similar vulnerabilities such as texture, background, and simplicity biases.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 22 Jul 2022.

Builds on

31 of 45 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2016Layer Normalizationcited by 5,579
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  9. 2019Invariant Risk Minimizationcited by 1,832
  10. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  11. 2019Natural Adversarial Examplescited by 1,401
  12. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
19 more
  1. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  2. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  3. 2021ImageNet-21K Pretraining for the Massescited by 607
  4. 2017Exploring Generalization in Deep Learningcited by 595
  5. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  6. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  7. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  8. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  9. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  10. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  11. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  12. 2021Vision Transformers are Robust Learnerscited by 208
  13. 2021Are Transformers More Robust Than CNNs?cited by 187
  14. 2019Ensemble Distribution Distillationcited by 158
  15. 2021Delving Deep into the Generalization of Vision Transformers under Distribution Shiftscited by 85
  16. 2021RobustART: Benchmarking Robustness on Architecture Design and Training Techniquescited by 78
  17. 2019Stable Rank Normalization for Improved Generalization in Neural Networks and GANscited by 41
  18. 2015Performance measures for classification systems with rejectioncited by 16
  19. 2021Exploring Corruption Robustness: Inductive Biases in Vision Transformers and MLP-Mixerscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 16; 2024: 7; 2025: 5; 2026: 2.

34 papers.

Sort:

By area:

  1. 2022A Light Recipe to Train Robust Vision Transformerscited by 64
  2. 2022MogaNet: Multi-order Gated Aggregation Networkcited by 59
  3. 2022ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasetscited by 44
  4. 2023Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Modelscited by 30
  5. 2023What do neural networks learn in image classification? A frequency shortcut perspectivecited by 28
  6. 2023Anatomy-aware and acquisition-agnostic joint registration with SynthMorphcited by 19
  7. 2023ConvNet vs Transformer, Supervised vs CLIP: Beyond ImageNet Accuracycited by 19
  8. 2022Not Just Pretty Pictures: Toward Interventional Data Augmentation Using Text-to-Image Generatorscited by 17
  9. 2023MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detectioncited by 17
  10. 2023Reliability in Semantic Segmentation: Are We on the Right Track?cited by 16
  11. 2023Reliable Evaluation of Adversarial Transferabilitycited by 12
  12. 2023On the unreasonable vulnerability of transformers for image restoration -- and an easy fixcited by 11
22 more
  1. 2024Are We Ready for Out-of-Distribution Detection in Digital Pathology?cited by 8
  2. 2023E-NER: Evidential Deep Learning for Trustworthy Named Entity Recognitioncited by 7
  3. 2023An Analysis of Initial Training Strategies for Exemplar-Free Class-Incremental Learningcited by 7
  4. 2024A Rate-Distortion View of Uncertainty Quantificationcited by 3
  5. 2023SkyGPT: Probabilistic Short-term Solar Forecasting Using Synthetic Sky Videos from Physics-constrained VideoGPTcited by 2
  6. 2024Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensemblingcited by 2
  7. 2024Situation Monitor: Diversity-Driven Zero-Shot Out-of-Distribution Detection using Budding Ensemble Architecture for Object Detectioncited by 2
  8. 2024On Evaluating Adversarial Robustness of Volumetric Medical Segmentation Modelscited by 2
  9. 2025Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated?cited by 2
  10. 2023Underwater target detection based on improved YOLOv7cited by 1
  11. 2023Hardwiring ViT Patch Selectivity into CNNs using Patch Mixingcited by 1
  12. 2023Exploring Non-additive Randomness on ViT against Query-Based Black-Box Attackscited by 1
  13. 2024GeoPos: A Minimal Positional Encoding for Enhanced Fine-Grained Details in Image Synthesis Using Convolutional Neural Networkscited by 1
  14. 2025Adversarial Robustness of Discriminative Self-Supervised Learning in Visioncited by 1
  15. 2023Fine-Grained ImageNet Classification in the Wildcited by 0
  16. 2023Improving Stain Invariance of CNNs for Segmentation by Fusing Channel Attention and Domain-Adversarial Trainingcited by 0
  17. 2024A Survey on Cell Nuclei Instance Segmentation and Classification: Leveraging Context and Attentioncited by 0
  18. 2025Divide and Conquer Self-Supervised Learning for High-Content Imagingcited by 0
  19. 2025SpaRTAN: Spatial Reinforcement Token-based Aggregation Network for Visual Recognitioncited by 0
  20. 2025DetailSemNet: Elevating Signature Verification through Detail-Semantic Integrationcited by 0
  21. 2026Anatomy of a failure: When, how, and why deep vision fails in scientific domainscited by 0
  22. 2026Vertical Fusion: Condensing Internal Representations for Robust ViT Classificationcited by 0