Model Generalization: A Sharpness Aware Optimization Perspective
Not on the map. Why?
Abstract (arXiv)
Sharpness-Aware Minimization (SAM) and adaptive sharpness-aware minimization (ASAM) aim to improve the model generalization. And in this project, we proposed three experiments to valid their generalization from the sharpness aware perspective. And our experiments show that sharpness aware-based optimization techniques could help to provide models with strong generalization ability. Our experiments also show that ASAM could improve the generalization performance on un-normalized data, but further research is needed to confirm this.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 7 of its 12 references that are in the map. First posted on arXiv 14 Aug 2022.
Builds on
7 of 12 references.
References · 7
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
- 2017Sharp Minima Can Generalize For Deep Netscited by 466
- 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
- 2021Sharpness-Aware Minimization Improves Language Model Generalizationcited by 108
- 2022Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Modelscited by 20
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.