mlsanityData to 7 Oct 2026

Fast Adversarial Training against Sparse Attacks Requires Loss Smoothing

arXiv 2502.21041 ↗First posted 28 Feb 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper studies fast adversarial training against sparse adversarial perturbations bounded by l0 norm. We demonstrate the challenges of employing 1-step attacks on l0 bounded perturbations for fast adversarial training, including degraded performance and the occurrence of catastrophic overfitting (CO). We highlight that CO in l0 adversarial training is caused by sub-optimal perturbation locations of 1-step attack. Theoretical and empirical analyses reveal that the loss landscape of l0 adversarial training is more craggy compared to its l_∞, l2 and l1 counterparts. Moreover, we corroborate that the craggy loss landscape can aggravate CO. To address these issues, we propose Fast-LS-l0 that incorporates soft labels and the trade-off loss function to smooth the adversarial loss landscape. Extensive experiments demonstrate our method can overcome the challenge of catastrophic overfitting, achieve state-of-the-art performance, and narrow down the performance gap between 1-step and multi-step adversarial training against sparse attacks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 48 of its 67 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

48 of 67 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  12. 2019Machine Unlearningcited by 1,020
36 more
  1. 2016Adversarial Machine Learning at Scalecited by 934
  2. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  3. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  4. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  5. 2019Adversarial Training for Free!cited by 638
  6. 2020Overfitting in adversarially robust deep learningcited by 504
  7. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  8. 2021Improving Robustness using Generated Datacited by 233
  9. 2020Understanding and Improving Fast Adversarial Trainingcited by 213
  10. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  11. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
  12. 2019Adversarial Training and Robustness for Multiple Perturbationscited by 182
  13. 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 181
  14. 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
  15. 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
  16. 2019Sparse and Imperceivable Adversarial Attackscited by 95
  17. 2018SparseFool: a few pixels make a big differencecited by 94
  18. 2018Hessian-based Analysis of Large Batch Training and Robustness to Adversariescited by 87
  19. 2020Understanding Catastrophic Overfitting in Single-step Adversarial Trainingcited by 75
  20. 2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 75
  21. 2020Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attackscited by 65
  22. 2022A Light Recipe to Train Robust Vision Transformerscited by 64
  23. 2020On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Themcited by 59
  24. 2023Decoupled Kullback-Leibler Divergence Losscited by 58
  25. 2019Efficient Adversarial Training with Transferable Adversarial Examplescited by 54
  26. 2021Mind the box: l1-APGD for sparse adversarial attacks on image classifierscited by 50
  27. 2022Make Some Noise: Reliable and Efficient Single-Step Adversarial Trainingcited by 42
  28. 2022FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfittingcited by 28
  29. 2022Fast Adversarial Training with Adaptive Step Sizecited by 24
  30. 2022Understanding and Combating Robust Overfitting via Input Loss Landscape Analysis and Regularizationcited by 22
  31. 2021Understanding Catastrophic Overfitting in Adversarial Trainingcited by 11
  32. 2021On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Trainingcited by 11
  33. 2024Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularizationcited by 11
  34. 2024Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependencycited by 9
  35. 2024σ-zero: Gradient-based Optimization of ℓ0-norm Adversarial Examplescited by 7
  36. 2023Adversarial Machine Learning for Social Good: Reframing the Adversary as an Allycited by 5

Built on by

Not cited yet.