# Model Generalization: A Sharpness Aware Optimization Perspective

arXiv: https://arxiv.org/abs/2208.06915 · HTML: https://mlsanity.com/paper/2208.06915/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 12 references that are in the map.
- First posted on arXiv 14 Aug 2022.

## Builds on (most cited first, 7 of 7)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Sharp Minima Can Generalize For Deep Nets](https://mlsanity.com/paper/1703.04933/) (2017)
- [ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks](https://mlsanity.com/paper/2102.11600/) (2021)
- [Sharpness-Aware Minimization Improves Language Model Generalization](https://mlsanity.com/paper/2110.08529/) (2021)
- [Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models](https://mlsanity.com/paper/2205.12694/) (2022)

## Built on by (most cited first, 2 of 2)

- [Multi-Dataset Co-Training with Sharpness-Aware Optimization for Audio Anti-spoofing](https://mlsanity.com/paper/2305.19953/) (2023)
- [Do Sharpness-based Optimizers Improve Generalization in Medical Image Analysis?](https://mlsanity.com/paper/2408.04065/) (2024)

## Abstract (arXiv)

Sharpness-Aware Minimization (SAM) and adaptive sharpness-aware minimization (ASAM) aim to improve the model generalization. And in this project, we proposed three experiments to valid their generalization from the sharpness aware perspective. And our experiments show that sharpness aware-based optimization techniques could help to provide models with strong generalization ability. Our experiments also show that ASAM could improve the generalization performance on un-normalized data, but further research is needed to confirm this.