# Fundamental Convergence Analysis of Sharpness-Aware Minimization

arXiv: https://arxiv.org/abs/2401.08060 · HTML: https://mlsanity.com/paper/2401.08060/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2024.

## Built on by (most cited first, 14 of 14)

- [Sharpness-Aware Minimization: General Analysis and Improved Rates](https://mlsanity.com/paper/2503.02225/) (2025)
- [SAMPa: Sharpness-aware Minimization Parallelized](https://mlsanity.com/paper/2410.10683/) (2024)
- [SAFE: Finding Sparse and Flat Minima to Improve Pruning](https://mlsanity.com/paper/2506.06866/) (2025)
- [SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation](https://mlsanity.com/paper/2502.18153/) (2025)
- [Bi-LoRA: Efficient Sharpness-Aware Minimization for Fine-Tuning Large-Scale Models](https://mlsanity.com/paper/2508.19564/) (2025)
- [Unveiling m-Sharpness Through the Structure of Stochastic Gradient Noise](https://mlsanity.com/paper/2509.18001/) (2025)
- [Sharpness-Aware Minimization with Adaptive Regularization for Training Deep Neural Networks](https://mlsanity.com/paper/2412.16854/) (2024)
- [Adaptive Sharpness-Aware Minimization with a Polyak-type Step size: A Theory-Grounded Scheduler](https://mlsanity.com/paper/2606.01827/) (2026)
- [Understanding SAM's Robustness to Noisy Labels through Gradient Down-weighting](https://mlsanity.com/paper/2411.17132/) (2024)
- [Sharpness-Aware Minimization with Z-Score Gradient Filtering](https://mlsanity.com/paper/2505.02369/) (2025)
- [Sharpness-Aware Minimization Can Hallucinate Minimizers](https://mlsanity.com/paper/2509.21818/) (2025)
- [Flatness-Aware Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2510.02174/) (2025)
- [Incorruptible Neural Networks: Training Models that can Generalize to Large Internal Perturbations](https://mlsanity.com/paper/2602.07320/) (2026)
- [A proximal subgradient method for nonconvex stochastic optimization under the Kurdyka-Lojasiewicz condition](https://mlsanity.com/paper/2608.05460/) (2026)

## Abstract (arXiv)

The paper investigates the fundamental convergence properties of Sharpness-Aware Minimization (SAM), a recently proposed gradient-based optimization method [Foret et al., 2021] that significantly improves the generalization of deep neural networks. The convergence properties, including the stationarity of accumulation points, the convergence of the sequence of gradients to the origin, the sequence of function values to the optimal value, and the sequence of iterates to the optimal solution, are established for the method. The universality of the provided convergence analysis, based on inexact gradient descent frameworks Khanh et al. [2023b], allows its extensions to efficient normalized versions of SAM such as F-SAM [Li et al., 2024], VaSSO [Li and Giannakis, 2023], RSAM [Liu et al., 2022], and to the unnormalized versions of SAM such as USAM [Andriushchenko and Flammarion, 2022]. Numerical experiments are conducted on classification tasks using deep learning models to confirm the practical aspects of our analysis.