# Sharpness-aware Quantization for Deep Neural Networks

arXiv: https://arxiv.org/abs/2111.12273 · HTML: https://mlsanity.com/paper/2111.12273/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 78 of its 109 references that are in the map.
- First posted on arXiv 24 Nov 2021.

## Builds on (most cited first, 30 of 78)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data](https://mlsanity.com/paper/1703.11008/) (2017)
- [Fantastic Generalization Measures and Where to Find Them](https://mlsanity.com/paper/1912.02178/) (2019)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [Learning Deconvolution Network for Semantic Segmentation](https://mlsanity.com/paper/1505.04366/) (2015)

## Built on by (most cited first, 25 of 25)

- [NIPQ: Noise proxy-based Integrated Pseudo-Quantization](https://mlsanity.com/paper/2206.00820/) (2022)
- [SDQ: Stochastic Differentiable Quantization with Mixed Precision](https://mlsanity.com/paper/2206.04459/) (2022)
- [Patch-wise Mixed-Precision Quantization of Vision Transformer](https://mlsanity.com/paper/2305.06559/) (2023)
- [Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMs](https://mlsanity.com/paper/2506.09104/) (2025)
- [SQuAT: Sharpness- and Quantization-Aware Training for BERT](https://mlsanity.com/paper/2210.07171/) (2022)
- [QGen: On the Ability to Generalize in Quantization Aware Training](https://mlsanity.com/paper/2404.11769/) (2024)
- [Lookbehind-SAM: k steps back, 1 step forward](https://mlsanity.com/paper/2307.16704/) (2023)
- [Sharpness-Aware Data Generation for Zero-shot Quantization](https://mlsanity.com/paper/2510.07018/) (2025)
- [Neural Networks with Quantization Constraints](https://mlsanity.com/paper/2210.15623/) (2022)
- [Efficient and Effective Methods for Mixed Precision Neural Network Quantization for Faster, Energy-efficient Inference](https://mlsanity.com/paper/2301.13330/) (2023)
- [Robust Training of Neural Networks at Arbitrary Precision and Sparsity](https://mlsanity.com/paper/2409.09245/) (2024)
- [The Split Matters: Flat Minima Methods for Improving the Performance of GNNs](https://mlsanity.com/paper/2306.09121/) (2023)
- [Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models](https://mlsanity.com/paper/2310.13315/) (2023)
- [Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective](https://mlsanity.com/paper/2509.00859/) (2025)
- [Zero-Shot Quantization via Weight-Space Arithmetic](https://mlsanity.com/paper/2604.03420/) (2026)
- [On Calibration of Modern Quantized Efficient Neural Networks](https://mlsanity.com/paper/2309.13866/) (2023)
- [AdaQAT: Adaptive Bit-Width Quantization-Aware Training](https://mlsanity.com/paper/2404.16876/) (2024)
- [Fisher-aware Quantization for DETR Detectors with Critical-category Objectives](https://mlsanity.com/paper/2407.03442/) (2024)
- [SAMSON: Sharpness-Aware Minimization Scaled by Outlier Normalization for Improving DNN Generalization and Robustness](https://mlsanity.com/paper/2211.11561/) (2022)
- [FlatENN: Train Flat for Enhanced Fault Tolerance of Quantized Deep Neural Networks](https://mlsanity.com/paper/2301.00675/) (2022)
- [CrAFT: Compression-Aware Fine-Tuning for Efficient Visual Task Adaptation](https://mlsanity.com/paper/2305.04526/) (2023)
- [Memory Faults in Activation-sparse Quantized Deep Neural Networks: Analysis and Mitigation using Sharpness-aware Training](https://mlsanity.com/paper/2406.10528/) (2024)
- [Nearly Lossless Adaptive Bit Switching](https://mlsanity.com/paper/2502.01199/) (2025)
- [Certification of Machine Learning Models via Directional Sharpness](https://mlsanity.com/paper/2606.25004/) (2026)
- [Neural Network Quantization by Learning Low-Loss Subspaces](https://mlsanity.com/paper/2606.25087/) (2026)

## Abstract (arXiv)

Network quantization is a dominant paradigm of model compression. However, the abrupt changes in quantized weights during training often lead to severe loss fluctuations and result in a sharp loss landscape, making the gradients unstable and thus degrading the performance. Recently, Sharpness-Aware Minimization (SAM) has been proposed to smooth the loss landscape and improve the generalization performance of the models. Nevertheless, directly applying SAM to the quantized models can lead to perturbation mismatch or diminishment issues, resulting in suboptimal performance. In this paper, we propose a novel method, dubbed Sharpness-Aware Quantization (SAQ), to explore the effect of SAM in model compression, particularly quantization for the first time. Specifically, we first provide a unified view of quantization and SAM by treating them as introducing quantization noises and adversarial perturbations to the model weights, respectively. According to whether the noise and perturbation terms depend on each other, SAQ can be formulated into three cases, which are analyzed and compared comprehensively. Furthermore, by introducing an efficient training strategy, SAQ only incurs a little additional training overhead compared with the default optimizer (e.g., SGD or AdamW). Extensive experiments on both convolutional neural networks and Transformers across various datasets (i.e., ImageNet, CIFAR-10/100, Oxford Flowers-102, Oxford-IIIT Pets) show that SAQ improves the generalization performance of the quantized models, yielding the SOTA results in uniform quantization. For example, on ImageNet, SAQ outperforms AdamW by 1.2% on the Top-1 accuracy for 4-bit ViT-B/16. Our 4-bit ResNet-50 surpasses the previous SOTA method by 0.9% on the Top-1 accuracy.