Data to 7 Oct 2026

Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models

arXiv 2303.01870 ↗First posted 3 Mar 2023cs.CV cs.CR cs.LGcited by 79

Not on the map. Why?

Abstract (arXiv)

While adversarial training has been extensively studied for ResNet architectures and low resolution datasets like CIFAR, much less is known for ImageNet. Given the recent debate about whether transformers are more robust than convnets, we revisit adversarial training on ImageNet comparing ViTs and ConvNeXts. Extensive experiments show that minor changes in architecture, most notably replacing PatchStem with ConvStem, and training scheme have a significant impact on the achieved robustness. These changes not only increase robustness in the seen ℓ_∞-threat model, but even more so improve generalization to unseen ℓ1/ℓ2-attacks. Our modified ConvNeXt, ConvNeXt + ConvStem, yields the most robust ℓ_∞-models across different ranges of model parameters and FLOPs, while our ViT + ConvStem yields the best generalization to unseen threat models.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Builds on 56 of its 71 references that are in the map. First posted on arXiv 3 Mar 2023.

Builds on

56 of 71 references.

References · 56

2017Attention Is All You Needcited by 59,932
2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
2020Training data-efficient image transformers & distillation through attentioncited by 4,560
2022A ConvNet for the 2020scited by 3,545
2016Gaussian Error Linear Units (GELUs)cited by 3,431
2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
2016Identity Mappings in Deep Residual Networkscited by 1,960

38 more

2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
2017Random Erasing Data Augmentationcited by 1,235
2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
2021MetaFormer Is Actually What You Need for Visioncited by 523
2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 479
2022MaxViT: Multi-Axis Vision Transformercited by 476
2022DeiT III: Revenge of the ViTcited by 417
2019Unlabeled Data Improves Adversarial Robustnesscited by 415
2021Early Convolutions Help Transformers See Bettercited by 395
2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
2021ResNet strikes back: An improved training procedure in timmcited by 365
2021XCiT: Cross-Covariance Image Transformerscited by 335
2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
2021Improving Robustness using Generated Datacited by 233
2019Fixing the train-test resolution discrepancycited by 209
2022Patches Are All You Need?cited by 201
2020Bag of Tricks for Adversarial Trainingcited by 199
2021Data Augmentation Can Improve Robustnesscited by 193
2021Are Transformers More Robust Than CNNs?cited by 187
2019Adversarial Training and Robustness for Multiple Perturbationscited by 182
2020Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuningcited by 152
2020Perceptual Adversarial Robustness: Defense Against Unseen Threat Modelscited by 137
2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 109
2020Smooth Adversarial Trainingcited by 89
2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
2020Do Wider Neural Networks Really Help Adversarial Robustness?cited by 79
2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
2022A Light Recipe to Train Robust Vision Transformerscited by 64
2022When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecturecited by 64
2021Mind the box: l1-APGD for sparse adversarial attacks on image classifierscited by 50
2022Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?cited by 47

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 18; 2024: 21; 2025: 26; 2026: 14.

79 papers.

Sort:

By area:

2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Modelscited by 119
2025Trading Inference-Time Compute for Adversarial Robustnesscited by 60
2023Robust Principles: Architectural Design Principles for Adversarially Robust CNNscited by 46
2024Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studiescited by 38
2024Revisiting Adversarial Training at Scalecited by 33
2023Expressive Losses for Verified Robustness via Convex Combinationscited by 23
2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
2023Instruct2Attack: Language-Guided Semantic Adversarial Attackscited by 16
2024MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifierscited by 16
2025One Surrogate to Fool Them All: Universal, Transferable, and Targeted Adversarial Attacks with CLIPcited by 15
2024Instant Adversarial Purification with Adversarial Consistency Distillationcited by 14
2023Towards Reliable Evaluation and Fast Training of Robust Semantic Segmentation Modelscited by 12

38 more

2023MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustnesscited by 12
2025CLIPure: Purification in Latent Space via CLIP for Adversarially Robust Zero-Shot Classificationcited by 12
2025Adversarially Robust CLIP Models Can Induce Better (Robust) Perceptual Metricscited by 10
2024FullLoRA: Efficiently Boosting the Robustness of Pretrained Vision Transformerscited by 9
2025AdvAD: Exploring Non-Parametric Diffusion for Imperceptible Adversarial Attackscited by 7
2025Diffusion-based Adversarial Purification from the Perspective of the Frequency Domaincited by 7
2023AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentationcited by 6
2023Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Modelscited by 6
2023Towards Evaluating Transfer-based Attacks Systematically, Practically, and Fairlycited by 6
2024Adversarial Training: A Surveycited by 6
2024Guardians of Image Quality: Benchmarking Defenses Against Adversarial Attacks on Image Quality Metricscited by 5
2024Towards Adversarial Robustness via Debiased High-Confidence Logit Alignmentcited by 5
2023OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shiftcited by 4
2023High-Performance Transformers for Table Structure Recognition Need Early Convolutionscited by 4
2023MVPatch: More Vivid Patch for Adversarial Camouflaged Attacks on Object Detectors in the Physical Worldcited by 4
2023Improving Transferability of Adversarial Examples via Bayesian Attackscited by 3
2024A Random Ensemble of Encrypted Vision Transformers for Adversarially Robust Defensecited by 3
2024Mitigating Low-Frequency Bias: Feature Recalibration and Frequency Attention Regularization for Adversarial Robustnesscited by 3
2024Towards Million-Scale Adversarial Robustness Evaluation With Stronger Individual Attackscited by 3
2025GreedyPixel: Fine-Grained Black-Box Adversarial Attack Via Greedy Algorithmcited by 3
2025Enhancing Diffusion-based Unrestricted Adversarial Attacks via Adversary Preferences Alignmentcited by 3
2025Boosting Generative Adversarial Transferability with Self-supervised Vision Transformer Featurescited by 3
2023Hindering Adversarial Attacks with Multiple Encrypted Patch Embeddingscited by 2
2023Exploring Geometry of Blind Spots in Vision Modelscited by 2
2024Assessing Visually-Continuous Corruption Robustness of Neural Networks Relative to Human Performancecited by 2
2024Improving Fast Adversarial Training via Self-Knowledge Guidancecited by 2
2024Robust image classification with multi-modal large language modelscited by 2
2024RobustBlack: Challenging Black-Box Adversarial Attacks on State-of-the-Art Defensescited by 2
2025Stop Walking in Circles! Bailing Out Early in Projected Gradient Descentcited by 2
2025FrameShield: Adversarially Robust Video Anomaly Detectioncited by 2
2023A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifierscited by 1
2023Efficient Key-Based Adversarial Defense for ImageNet by Using Pre-trained Modelcited by 1
2023Topology-preserving Adversarial Training for Alleviating Natural Accuracy Degradationcited by 1
2024MALT Powers Up Adversarial Attackscited by 1
2024On the Adversarial Transferability of Generalized "Skip Connections"cited by 1
2024Conflict-Aware Adversarial Trainingcited by 1
2025Beyond Accuracy: What Matters in Designing Well-Behaved Image Classification Models?cited by 1
2025DBLP: Noise Bridge Consistency Distillation For Efficient And Reliable Adversarial Purificationcited by 1

50 most cited of 79 All 79 (JSON)