# Overcoming Oscillations in Quantization-Aware Training

arXiv: https://arxiv.org/abs/2203.11086 · HTML: https://mlsanity.com/paper/2203.11086/

## Key facts

- Cited by 119 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 39 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 25 of 25)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [A White Paper on Neural Network Quantization](https://mlsanity.com/paper/2106.08295/) (2021)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [Deep Learning with Limited Numerical Precision](https://mlsanity.com/paper/1502.02551/) (2015)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)
- [Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks](https://mlsanity.com/paper/1908.05033/) (2019)
- [Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets](https://mlsanity.com/paper/1903.05662/) (2019)
- [LSQ+: Improving low-bit quantization through learnable offsets and better initialization](https://mlsanity.com/paper/2004.09576/) (2020)
- [Quantization Networks](https://mlsanity.com/paper/1911.09464/) (2019)
- [Training with Quantization Noise for Extreme Model Compression](https://mlsanity.com/paper/2004.07320/) (2020)
- [Deep Learning with Low Precision by Half-wave Gaussian Quantization](https://mlsanity.com/paper/1702.00953/) (2017)
- [Loss Aware Post-training Quantization](https://mlsanity.com/paper/1911.07190/) (2019)
- [Network Quantization with Element-wise Gradient Scaling](https://mlsanity.com/paper/2104.00903/) (2021)
- [Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization](https://mlsanity.com/paper/1906.02107/) (2019)
- [PROFIT: A Novel Training Method for sub-4-bit MobileNet Models](https://mlsanity.com/paper/2008.04693/) (2020)
- [Differentiable Model Compression via Pseudo Quantization Noise](https://mlsanity.com/paper/2104.09987/) (2021)
- [Probabilistic Binary Neural Networks](https://mlsanity.com/paper/1809.03368/) (2018)
- [Quantization-Guided Training for Compact TinyML Models](https://mlsanity.com/paper/2103.06231/) (2021)

## Built on by (most cited first, 30 of 119)

- [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024)
- [On-Device Language Models: A Comprehensive Review](https://mlsanity.com/paper/2409.00088/) (2024)
- [FP8 Quantization: The Power of the Exponent](https://mlsanity.com/paper/2208.09225/) (2022)
- [EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models](https://mlsanity.com/paper/2310.03270/) (2023)
- [Oscillation-free Quantization for Low-bit Vision Transformers](https://mlsanity.com/paper/2302.02210/) (2023)
- [Pruning vs Quantization: Which is Better?](https://mlsanity.com/paper/2307.02973/) (2023)
- [NIPQ: Noise proxy-based Integrated Pseudo-Quantization](https://mlsanity.com/paper/2206.00820/) (2022)
- [FP8 versus INT8 for efficient deep learning inference](https://mlsanity.com/paper/2303.17951/) (2023)
- [A Machine Learning-oriented Survey on Tiny Machine Learning](https://mlsanity.com/paper/2309.11932/) (2023)
- [ApiQ: Finetuning of 2-Bit Quantized Large Language Model](https://mlsanity.com/paper/2402.05147/) (2024)
- [A Practical Mixed Precision Algorithm for Post-Training Quantization](https://mlsanity.com/paper/2302.05397/) (2023)
- [EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Models](https://mlsanity.com/paper/2401.04585/) (2024)
- [Model-Preserving Adaptive Rounding](https://mlsanity.com/paper/2505.22988/) (2025)
- [QVGen: Pushing the Limit of Quantized Video Generative Models](https://mlsanity.com/paper/2505.11497/) (2025)
- [SLA2: Sparse-Linear Attention with Learnable Routing and QAT](https://mlsanity.com/paper/2602.12675/) (2026)
- [Post-training Quantization for Text-to-Image Diffusion Models with Progressive Calibration and Activation Relaxing](https://mlsanity.com/paper/2311.06322/) (2023)
- [Oscillation-Reduced MXFP4 Training for Vision Transformers](https://mlsanity.com/paper/2502.20853/) (2025)
- [DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformers](https://mlsanity.com/paper/2408.03291/) (2024)
- [Genie: Show Me the Data for Quantization](https://mlsanity.com/paper/2212.04780/) (2022)
- [Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precision](https://mlsanity.com/paper/2307.00331/) (2023)
- [FPTQuant: Function-Preserving Transforms for LLM Quantization](https://mlsanity.com/paper/2506.04985/) (2025)
- [Softmax Bias Correction for Quantized Generative Models](https://mlsanity.com/paper/2309.01729/) (2023)
- [ESPACE: Dimensionality Reduction of Activations for Model Compression](https://mlsanity.com/paper/2410.05437/) (2024)
- [TetraJet-v2: Accurate NVFP4 Training for Large Language Models with Oscillation Suppression and Outlier Control](https://mlsanity.com/paper/2510.27527/) (2025)
- [Retraining-free Model Quantization via One-Shot Weight-Coupling Learning](https://mlsanity.com/paper/2401.01543/) (2024)
- [CacheQuant: Comprehensively Accelerated Diffusion Models](https://mlsanity.com/paper/2503.01323/) (2025)
- [Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices](https://mlsanity.com/paper/2206.10844/) (2022)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [EQ-Net: Elastic Quantization Neural Networks](https://mlsanity.com/paper/2308.07650/) (2023)
- [Exploring Quantization for Efficient Pre-Training of Transformer Language Models](https://mlsanity.com/paper/2407.11722/) (2024)

## Abstract (arXiv)

When training neural networks with simulated quantization, we observe that quantized weights can, rather unexpectedly, oscillate between two grid-points. The importance of this effect and its impact on quantization-aware training (QAT) are not well-understood or investigated in literature. In this paper, we delve deeper into the phenomenon of weight oscillations and show that it can lead to a significant accuracy degradation due to wrongly estimated batch-normalization statistics during inference and increased noise during training. These effects are particularly pronounced in low-bit (≤ 4-bits) quantization of efficient networks with depth-wise separable layers, such as MobileNets and EfficientNets. In our analysis we investigate several previously proposed QAT algorithms and show that most of these are unable to overcome oscillations. Finally, we propose two novel QAT algorithms to overcome oscillations during training: oscillation dampening and iterative weight freezing. We demonstrate that our algorithms achieve state-of-the-art accuracy for low-bit (3 & 4 bits) weight and activation quantization of efficient architectures, such as MobileNetV2, MobileNetV3, and EfficentNet-lite on ImageNet. Our source code is available at https://github.com/qualcomm-ai-research/oscillations-qat.