# SALR: Sharpness-aware Learning Rate Scheduler for Improved Generalization

arXiv: https://arxiv.org/abs/2011.05348 · HTML: https://mlsanity.com/paper/2011.05348/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2020.

## Built on by (most cited first, 5 of 5)

- [Good regularity creates large learning rate implicit biases: edge of stability, balancing, and catapult](https://mlsanity.com/paper/2310.17087/) (2023)
- [Bilateral Sharpness-Aware Minimization for Flatter Minima](https://mlsanity.com/paper/2409.13173/) (2024)
- [Wolkowicz-Styan Upper Bound on the Hessian Eigenspectrum for Cross-Entropy Loss in Nonlinear Smooth Neural Networks](https://mlsanity.com/paper/2604.10202/) (2026)
- [Adaptively Sampling-Reusing-Mixing Decomposed Gradients to Speed Up Sharpness Aware Minimization](https://mlsanity.com/paper/2510.03763/) (2025)
- [Closed-Form Steepest Descent Direction toward Flat Minima: Reducing Upper Bounds on the Loss Hessian Eigenspectrum in Neural Networks](https://mlsanity.com/paper/2606.28662/) (2026)

## Abstract (arXiv)

In an effort to improve generalization in deep learning and automate the process of learning rate scheduling, we propose SALR: a sharpness-aware learning rate update technique designed to recover flat minimizers. Our method dynamically updates the learning rate of gradient-based optimizers based on the local sharpness of the loss function. This allows optimizers to automatically increase learning rates at sharp valleys to increase the chance of escaping them. We demonstrate the effectiveness of SALR when adopted by various algorithms over a broad range of networks. Our experiments indicate that SALR improves generalization, converges faster, and drives solutions to significantly flatter regions.