mlsanityData to 14 Oct 2026

Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization

arXiv 2605.10183 ↗First posted 11 May 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Sharpness-Aware Minimization (SAM) improves generalization by minimizing the worst-case loss within a fixed parameter-space radius neighborhood. SAM and its variants mainly rely on a first-order linearized surrogate, while flat minima are inherently a second-order (curvature) notion.We revisit this mismatch and propose Loss-Equated SAM (LE-SAM), which inverts the traditional SAM mechanism that fixed perturbation radius with a fixed loss-space budget,effectively removing gradient-norm-dominated learning signals and shifting optimization toward curvature-dominated terms. Extensive experiments across diverse benchmarks and tasks demonstrate the strong generalization ability of LESAM that consistently outperforms SAM and even its variants, achieving the state-of-the-art performance.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 58 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

41 of 58 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  6. 2016Wide Residual Networkscited by 3,070
  7. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  8. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  9. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  10. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  11. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
  12. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
29 more
  1. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  2. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
  3. 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 292
  4. 2022Surrogate Gap Minimization Improves Sharpness-Aware Trainingcited by 172
  5. 2016Deep Pyramidal Residual Networkscited by 163
  6. 2020Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergencecited by 137
  7. 2021Efficient Sharpness-aware Minimization for Improved Training of Neural Networkscited by 137
  8. 2021Targeted Supervised Contrastive Learning for Long-Tailed Recognitioncited by 132
  9. 2021Contrastive Learning based Hybrid Networks for Long-Tailed Image Classificationcited by 130
  10. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  11. 2019Revisiting the Polyak step sizecited by 55
  12. 2024Friendly Sharpness-Aware Minimizationcited by 53
  13. 2023Global and Local Mixture Consistency Cumulative Learning for Long-tailed Visual Recognitionscited by 43
  14. 2022Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solutioncited by 40
  15. 2022Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscapecited by 39
  16. 2024On the Duality Between Sharpness-Aware Minimization and Adversarial Trainingcited by 24
  17. 2024Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late in Trainingcited by 24
  18. 2023Adaptive SGD with Polyak stepsize and Line-search: Robust Convergence and Variance Reductioncited by 23
  19. 2025Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimizationcited by 21
  20. 2023ERM++: An Improved Baseline for Domain Generalizationcited by 18
  21. 2023Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Termcited by 16
  22. 2024Momentum-SAM: Sharpness Aware Minimization without Computational Overheadcited by 14
  23. 2024A Universal Class of Sharpness-Aware Minimization Algorithmscited by 14
  24. 2025Focal-SAM: Focal Sharpness-Aware Minimization for Long-Tailed Classificationcited by 14
  25. 2024Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performancecited by 13
  26. 2024Stabilizing Sharpness-aware Minimization Through A Simple Renormalization Strategycited by 8
  27. 2020Complexity Guarantees for Polyak Steps with Momentumcited by 7
  28. 2021Sharpness-Aware Minimization with Dynamic Reweightingcited by 6
  29. 2024Exact Convergence rate of the subgradient method by using Polyak step sizecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Action-Consequence Alignment for Reliable Planning and Self-Improving in Latent World Modelscited by 0