mlsanityData to 14 Oct 2026

Adversarial Robustness against Multiple and Single lp-Threat Models via Quick Fine-Tuning of Robust Classifiers

arXiv 2105.12508 ↗First posted 26 May 2021cs.LG cs.CR cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

A major drawback of adversarially robust models, in particular for large scale datasets like ImageNet, is the extremely long training time compared to standard ones. Moreover, models should be robust not only to one lp-threat model but ideally to all of them. In this paper we propose Extreme norm Adversarial Training (E-AT) for multiple-norm robustness which is based on geometric properties of lp-balls. E-AT costs up to three times less than other adversarial training methods for multiple-norm robustness. Using E-AT we show that for ImageNet a single epoch and for CIFAR-10 three epochs are sufficient to turn any lp-robust model into a multiple-norm robust model. In this way we get the first multiple-norm robust model for ImageNet and boost the state-of-the-art for multiple-norm robustness to more than 51% on CIFAR-10. Finally, we study the general transfer via fine-tuning of adversarial robustness between different individual lp-threat models and improve the previous SOTA l1-robustness on both CIFAR-10 and ImageNet. Extensive experiments show that our scheme works across datasets and architectures including vision transformers.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 50 of its 67 references that are in the map. First posted on arXiv 26 May 2021.

Builds on

50 of 67 references.

References · 50

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  2. 2013Intriguing properties of neural networkscited by 5,155
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  4. 2016Wide Residual Networkscited by 3,070
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  7. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  8. 2016Adversarial examples in the physical worldcited by 1,769
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  10. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  12. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
38 more
  1. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 661
  2. 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
  3. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
  4. 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
  5. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  6. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
  7. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 328
  8. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 278
  9. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
  10. 2019Adversarial Training and Robustness for Multiple Perturbationscited by 183
  11. 2018Disentangling Adversarial Robustness and Generalizationcited by 172
  12. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 172
  13. 2017EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examplescited by 155
  14. 2020Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuningcited by 153
  15. 2018Scaling provable adversarial defensescited by 143
  16. 2020Perceptual Adversarial Robustness: Defense Against Unseen Threat Modelscited by 137
  17. 2018Query-Efficient Hard-label Black-box Attack:An Optimization-based Approachcited by 130
  18. 2018Towards the first adversarially robust neural network model on MNISTcited by 104
  19. 2018SparseFool: a few pixels make a big differencecited by 94
  20. 2019Adversarial Robustness Against the Union of Multiple Perturbation Modelscited by 88
  21. 2020Do Wider Neural Networks Really Help Adversarial Robustness?cited by 79
  22. 2020Adversarial Robustness on In- and Out-Distribution Improves Explainabilitycited by 73
  23. 2020Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attackscited by 65
  24. 2021Mind the box: l1-APGD for sparse adversarial attacks on image classifierscited by 51
  25. 2020Adversarial Training against Location-Optimized Adversarial Patchescited by 50
  26. 2020Learning perturbation sets for robust machine learningcited by 48
  27. 2019An Alternative Surrogate Loss for PGD-based Adversarial Testingcited by 45
  28. 2019Provable robustness against all adversarial lp-perturbations for p≥ 1cited by 39
  29. 2020A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuningcited by 30
  30. 2020Augmented Lagrangian Adversarial Attackscited by 28
  31. 2018Logit Pairing Methods Can Fool Gradient-Based Attackscited by 27
  32. 2019Transfer of Adversarial Robustness Between Perturbation Typescited by 25
  33. 2019On the Design of Black-box Adversarial Examples by Leveraging Gradient-free Optimization and Operator Splitting Methodcited by 21
  34. 2019Yet another but more efficient black-box adversarial attack: tiling and evolution strategiescited by 17
  35. 2020Learning to Generate Noise for Multi-Attack Robustnesscited by 17
  36. 2018Adversarial Framing for Image and Video Classificationcited by 15
  37. 2020Adversarial Momentum-Contrastive Pre-Trainingcited by 10
  38. 2019There are No Bit Parts for Sign Bits in Black-Box Attackscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 6; 2025: 5; 2026: 5.

23 papers.

Sort:

By area:

  1. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  2. 2023Seasoning Model Soups for Robustness to Adversarial and Natural Distribution Shiftscited by 23
  3. 2023LION: Implicit Vision Prompt Tuningcited by 22
  4. 2023Latent Diffusion Counterfactual Explanationscited by 17
  5. 2023OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shiftcited by 4
  6. 2024Fine-Tuning Text-To-Image Diffusion Models for Class-Wise Spurious Feature Generationcited by 4
  7. 2024A Unified Understanding of Adversarial Vulnerability Regarding Unimodal Models and Vision-Language Pre-training Modelscited by 4
  8. 2023Exploring Geometry of Blind Spots in Vision Modelscited by 2
  9. 2025Adapting to Evolving Adversaries with Regularized Continual Robust Trainingcited by 2
  10. 2025Evading Data Provenance in Deep Neural Networkscited by 2
  11. 2024RAMP: Boosting Adversarial Robustness Against Multiple lp Perturbations for Universal Robustnesscited by 1
  12. 2024Position: Towards Resilience Against Adversarial Examplescited by 1
11 more
  1. 2024Deep Adversarial Defense Against Multilevel-Lp Attackscited by 1
  2. 2024Towards Generalized Certified Robustness with Multi-Norm Trainingcited by 1
  3. 2025A Guide to Robust Generalization: The Impact of Architecture, Pre-training, and Optimization Strategycited by 1
  4. 2025Generalist++: A Meta-learning Framework for Mitigating Trade-off in Adversarial Trainingcited by 1
  5. 2026When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Studycited by 1
  6. 2023Bridging Models to Defend: A Population-Based Strategy for Robust Adversarial Defensecited by 0
  7. 2025Stabilizing Multi-Attack Adversarial Training via Bandit Optimizationcited by 0
  8. 2026S2O: Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 0
  9. 2026TaFD: Threat-Aware Frequency Decoupling for Adversarial Robustness against Heterogeneous Attackscited by 0
  10. 2026RoME: Robust Mixture of Low-Rank Experts against Multiple Adversarial Perturbationscited by 0
  11. 2026Robust CurveMoE: Multi-Norm Adversarial Defense for Mixture-of-Experts Models via Mode Connectivitycited by 0