LSAM: Asynchronous Distributed Training with Landscape-Smoothed Sharpness-Aware Minimization
Not on the map. Why?
Abstract (arXiv)
While Sharpness-Aware Minimization (SAM) improves generalization in deep neural networks by minimizing both loss and sharpness, it suffers from inefficiency in distributed large-batch training. We present Landscape-Smoothed SAM (LSAM), a novel optimizer that preserves SAM's generalization advantages while offering superior efficiency. LSAM integrates SAM's adversarial steps with an asynchronous distributed sampling strategy, generating an asynchronous distributed sampling scheme, producing a smoothed sharpness-aware loss landscape for optimization. This design eliminates synchronization bottlenecks, accelerates large-batch convergence, and delivers higher final accuracy compared to data-parallel SAM.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 54 references that are in the map. First posted on arXiv 3 Sep 2025.
Builds on
34 of 54 references.
References · 34
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
- 2016Wide Residual Networkscited by 3,070
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2017Exploring Generalization in Deep Learningcited by 600
- 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 359
- 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
- 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 288
- 2022Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Modelscited by 174
22 more
- 2022Towards Understanding Sharpness-Aware Minimizationcited by 146
- 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
- 2021Efficient Sharpness-aware Minimization for Improved Training of Neural Networkscited by 137
- 2022Towards Efficient and Scalable Sharpness-Aware Minimizationcited by 129
- 2014Deep learning with Elastic Averaging SGDcited by 90
- 2020Structured Logconcave Sampling with a Restricted Gaussian Oraclecited by 86
- 2022Improved analysis for a proximal algorithm for samplingcited by 73
- 2024Friendly Sharpness-Aware Minimizationcited by 53
- 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
- 2023Reverse Diffusion Monte Carlocited by 41
- 2020Nonlinear Two-Time-Scale Stochastic Approximation: Convergence and Finite-Time Performancecited by 35
- 2024Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimizationcited by 28
- 2024Diffusive Gibbs Samplingcited by 21
- 2023Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optimacited by 20
- 2023Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Termcited by 16
- 2023mSAM: Micro-Batch-Averaged Sharpness-Aware Minimizationcited by 10
- 2024Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learningcited by 8
- 2019Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models: Extensioncited by 7
- 2023Entropy-MCMC: Sampling from Flat Basins with Easecited by 6
- 2024GRAWA: Gradient-based Weighted Averaging for Distributed Training of Deep Learning Modelscited by 4
- 2017Parle: parallelizing stochastic gradient descentcited by 2
- 2025Asynchronous Sharpness-Aware Minimization For Fast and Accurate Deep Learningcited by 2
Built on by
Not cited yet.