mlsanityData to 7 Oct 2026

Overcoming Oscillations in Quantization-Aware Training

arXiv 2203.11086 ↗First posted 21 Mar 2022cs.LGcited by 116

Not on the map. Why?

Abstract (arXiv)

When training neural networks with simulated quantization, we observe that quantized weights can, rather unexpectedly, oscillate between two grid-points. The importance of this effect and its impact on quantization-aware training (QAT) are not well-understood or investigated in literature. In this paper, we delve deeper into the phenomenon of weight oscillations and show that it can lead to a significant accuracy degradation due to wrongly estimated batch-normalization statistics during inference and increased noise during training. These effects are particularly pronounced in low-bit (≤ 4-bits) quantization of efficient networks with depth-wise separable layers, such as MobileNets and EfficientNets. In our analysis we investigate several previously proposed QAT algorithms and show that most of these are unable to overcome oscillations. Finally, we propose two novel QAT algorithms to overcome oscillations during training: oscillation dampening and iterative weight freezing. We demonstrate that our algorithms achieve state-of-the-art accuracy for low-bit (3 & 4 bits) weight and activation quantization of efficient architectures, such as MobileNetV2, MobileNetV3, and EfficentNet-lite on ImageNet. Our source code is available at https://github.com/qualcomm-ai-research/oscillations-qat.

Key facts. Cited by 116 papers in the map (data through 7 Oct 2026). Builds on 25 of its 39 references that are in the map. First posted on arXiv 21 Mar 2022.

Builds on

25 of 39 references.

References · 25

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  4. 2019Learned Step Size Quantizationcited by 604
  5. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  6. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  7. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  8. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  9. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  10. 2021A White Paper on Neural Network Quantizationcited by 453
  11. 2015Deep Learning with Limited Numerical Precisioncited by 368
  12. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
13 more
  1. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  2. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 225
  3. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 180
  4. 2019Quantization Networkscited by 151
  5. 2020Training with Quantization Noise for Extreme Model Compressioncited by 145
  6. 2017Deep Learning with Low Precision by Half-wave Gaussian Quantizationcited by 115
  7. 2019Loss Aware Post-training Quantizationcited by 94
  8. 2021Network Quantization with Element-wise Gradient Scalingcited by 82
  9. 2019Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimizationcited by 58
  10. 2020PROFIT: A Novel Training Method for sub-4-bit MobileNet Modelscited by 50
  11. 2021Differentiable Model Compression via Pseudo Quantization Noisecited by 44
  12. 2018Probabilistic Binary Neural Networkscited by 20
  13. 2021Quantization-Guided Training for Compact TinyML Modelscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 20; 2024: 31; 2025: 39; 2026: 18.

116 papers.

Sort:

By area:

  1. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  2. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  3. 2022FP8 Quantization: The Power of the Exponentcited by 87
  4. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Modelscited by 77
  5. 2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 55
  6. 2023Pruning vs Quantization: Which is Better?cited by 45
  7. 2022NIPQ: Noise proxy-based Integrated Pseudo-Quantizationcited by 35
  8. 2023FP8 versus INT8 for efficient deep learning inferencecited by 31
  9. 2024ApiQ: Finetuning of 2-Bit Quantized Large Language Modelcited by 27
  10. 2023A Machine Learning-oriented Survey on Tiny Machine Learningcited by 26
  11. 2023A Practical Mixed Precision Algorithm for Post-Training Quantizationcited by 23
  12. 2024EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Modelscited by 23
38 more
  1. 2023Post-training Quantization for Text-to-Image Diffusion Models with Progressive Calibration and Activation Relaxingcited by 21
  2. 2026SLA2: Sparse-Linear Attention with Learnable Routing and QATcited by 21
  3. 2025Oscillation-Reduced MXFP4 Training for Vision Transformerscited by 20
  4. 2025QVGen: Pushing the Limit of Quantized Video Generative Modelscited by 20
  5. 2025Model-Preserving Adaptive Roundingcited by 20
  6. 2024DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformerscited by 19
  7. 2022Genie: Show Me the Data for Quantizationcited by 17
  8. 2023Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precisioncited by 17
  9. 2025FPTQuant: Function-Preserving Transforms for LLM Quantizationcited by 16
  10. 2023Softmax Bias Correction for Quantized Generative Modelscited by 15
  11. 2024ESPACE: Dimensionality Reduction of Activations for Model Compressioncited by 15
  12. 2024Retraining-free Model Quantization via One-Shot Weight-Coupling Learningcited by 13
  13. 2025CacheQuant: Comprehensively Accelerated Diffusion Modelscited by 13
  14. 2025TetraJet-v2: Accurate NVFP4 Training for Large Language Models with Oscillation Suppression and Outlier Controlcited by 13
  15. 2022Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devicescited by 12
  16. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  17. 2023EQ-Net: Elastic Quantization Neural Networkscited by 10
  18. 2024Exploring Quantization for Efficient Pre-Training of Transformer Language Modelscited by 10
  19. 2023MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantizationcited by 7
  20. 2025Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMscited by 7
  21. 2022OLLA: Optimizing the Lifetime and Location of Arrays to Reduce the Memory Usage of Neural Networkscited by 6
  22. 2023Reducing the Side-Effects of Oscillations in Training of Quantized YOLO Networkscited by 6
  23. 2024Custom Gradient Estimators are Straight-Through Estimators in Disguisecited by 6
  24. 2025Diffusion Model Quantization: A Reviewcited by 6
  25. 2022DenseShift: Towards Accurate and Efficient Low-Bit Power-of-Two Quantizationcited by 5
  26. 2023EPTQ: Enhanced Post-Training Quantization via Hessian-guided Network-wise Optimizationcited by 5
  27. 2024Asymptotic Unbiased Sample Sampling to Speed Up Sharpness-Aware Minimizationcited by 5
  28. 2024Quantized Prompt for Efficient Generalization of Vision-Language Modelscited by 5
  29. 2025BHViT: Binarized Hybrid Vision Transformercited by 5
  30. 2025CASP: Compression of Large Multimodal Models Based on Attention Sparsitycited by 5
  31. 2025Improving the Straight-Through Estimator with Zeroth-Order Informationcited by 5
  32. 2022Neural Networks with Quantization Constraintscited by 4
  33. 2023QBitOpt: Fast and Accurate Bitwidth Reallocation during Trainingcited by 4
  34. 2024Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulatorscited by 4
  35. 2024SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generatorscited by 4
  36. 2025Modulated Diffusion: Accelerating Generative Modeling with Modulated Quantizationcited by 4
  37. 2025CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Trainingcited by 4
  38. 2024Oh! We Freeze: Improving Quantized Knowledge Distillation via Signal Propagation Analysis for Large Language Modelscited by 3

50 most cited of 116 All 116 (JSON)