mlsanityData to 7 Oct 2026

On the Duality Between Sharpness-Aware Minimization and Adversarial Training

arXiv 2402.15152 ↗First posted 23 Feb 2024cs.LG cs.AI cs.CRcited by 24

Not on the map. Why?

Abstract (arXiv)

Adversarial Training (AT), which adversarially perturb the input samples during training, has been acknowledged as one of the most effective defenses against adversarial attacks, yet suffers from inevitably decreased clean accuracy. Instead of perturbing the samples, Sharpness-Aware Minimization (SAM) perturbs the model weights during training to find a more flat loss landscape and improve generalization. However, as SAM is designed for better clean accuracy, its effectiveness in enhancing adversarial robustness remains unexplored. In this work, considering the duality between SAM and AT, we investigate the adversarial robustness derived from SAM. Intriguingly, we find that using SAM alone can improve adversarial robustness. To understand this unexpected property of SAM, we first provide empirical and theoretical insights into how SAM can implicitly learn more robust features, and conduct comprehensive experiments to show that SAM can improve adversarial robustness notably without sacrificing any clean accuracy, shedding light on the potential of SAM to be a substitute for AT when accuracy comes at a higher priority. Code is available at https://github.com/weizeming/SAM_AT.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 64 of its 80 references that are in the map. First posted on arXiv 23 Feb 2024.

Builds on

64 of 80 references.

References · 64

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  8. 2016Wide Residual Networkscited by 3,057
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  10. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  11. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  12. 2016Identity Mappings in Deep Residual Networkscited by 1,960
38 more
  1. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  2. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  3. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  4. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  5. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  6. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  7. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  8. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  9. 2018Robustness May Be at Odds with Accuracycited by 873
  10. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  11. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  12. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  13. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  14. 2019Adversarial Training for Free!cited by 638
  15. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  16. 2017Exploring Generalization in Deep Learningcited by 595
  17. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  18. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  19. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
  20. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  21. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  22. 2018Feature Denoising for Improving Adversarial Robustnesscited by 312
  23. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  24. 2020Bag of Tricks for Adversarial Trainingcited by 199
  25. 2018Spatially Transformed Adversarial Examplescited by 195
  26. 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 177
  27. 2020Torchattacks: A PyTorch Repository for Adversarial Attackscited by 156
  28. 2022Towards Understanding Sharpness-Aware Minimizationcited by 145
  29. 2020To be Robust or to be Fair: Towards Fairness in Adversarial Trainingcited by 137
  30. 2021Efficient Sharpness-aware Minimization for Improved Training of Neural Networkscited by 137
  31. 2022Towards Efficient and Scalable Sharpness-Aware Minimizationcited by 129
  32. 2021Sharpness-Aware Minimization Improves Language Model Generalizationcited by 108
  33. 2023Jatmo: Prompt Injection Defense by Task-Specific Finetuningcited by 89
  34. 2023Rethinking Model Ensemble in Transfer-based Adversarial Attackscited by 88
  35. 2022Fisher SAM: Information Geometry and Sharpness Aware Minimisationcited by 84
  36. 2023Robust Classification via a Single Diffusion Modelcited by 79
  37. 2022Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approachcited by 70
  38. 2022When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecturecited by 64

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 13; 2026: 8.

24 papers.

Sort:

By area:

  1. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyondcited by 76
  2. 2024Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late in Trainingcited by 24
  3. 2024Exploring the Robustness of In-Context Learning with Noisy Labelscited by 13
  4. 2025Unveiling the Basin-Like Loss Landscape in Large Language Modelscited by 13
  5. 2024Adversarial Representation Engineering: A General Model Editing Framework for Large Language Modelscited by 12
  6. 2025Sharpness-Aware Minimization: General Analysis and Improved Ratescited by 10
  7. 2025SAFE: Finding Sparse and Flat Minima to Improve Pruningcited by 5
  8. 2026Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minimacited by 4
  9. 2025Dual-Space Smoothness for Robust and Balanced LLM Unlearningcited by 2
  10. 2025A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Biascited by 2
  11. 2026Reward Sharpness-Aware Fine-Tuning for Diffusion Modelscited by 2
  12. 2025A Flat Minima Perspective on Understanding Augmentations and Model Robustnesscited by 1
12 more
  1. 2025Flatness-aware Curriculum Learning via Adversarial Difficultycited by 1
  2. 2026When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Studycited by 1
  3. 2025Feature Statistics with Uncertainty Help Adversarial Robustnesscited by 0
  4. 2025Understanding Machine Unlearning Through the Lens of Mode Connectivitycited by 0
  5. 2025Identifying and Understanding Cross-Class Features in Adversarial Trainingcited by 0
  6. 2025On the Escaping Efficiency of Distributed Adversarial Training Algorithmscited by 0
  7. 2025S2AP: Score-space Sharpness Minimization for Adversarial Pruningcited by 0
  8. 2026Adversarial Samples Are Not Created Equalcited by 0
  9. 2026UTOPIA: Unlearnable Tabular Data via Decoupled Shortcut Embeddingcited by 0
  10. 2026Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimizationcited by 0
  11. 2026Understanding Machine Unlearning Through the Lens of Mode Connectivitycited by 0
  12. 2026Sparsity-Adaptive Sharpness-Aware Minimizationcited by 0