# QReg: On Regularization Effects of Quantization

arXiv: https://arxiv.org/abs/2206.12372 · HTML: https://mlsanity.com/paper/2206.12372/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 24 Jun 2022.

## Builds on (most cited first, 16 of 16)

- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [Towards Universal Object Detection by Domain Attention](https://mlsanity.com/paper/1904.04402/) (2019)
- [Loss-aware Binarization of Deep Networks](https://mlsanity.com/paper/1611.01600/) (2016)
- [Alternating Multi-bit Quantization for Recurrent Neural Networks](https://mlsanity.com/paper/1802.00150/) (2018)
- [Loss-aware Weight Quantization of Deep Networks](https://mlsanity.com/paper/1802.08635/) (2018)
- [Pareto-Optimal Quantized ResNet Is Mostly 4-bit](https://mlsanity.com/paper/2105.03536/) (2021)
- [Quantization of Fully Convolutional Networks for Accurate Biomedical Image Segmentation](https://mlsanity.com/paper/1803.04907/) (2018)
- [Deep neural networks are robust to weight binarization and other non-linear distortions](https://mlsanity.com/paper/1606.01981/) (2016)
- [Quantization of Deep Neural Networks for Accurate Edge Computing](https://mlsanity.com/paper/2104.12046/) (2021)
- [Why Quantization Improves Generalization: NTK of Binary Weight Neural Networks](https://mlsanity.com/paper/2206.05916/) (2022)
- [PBGen: Partial Binarization of Deconvolution-Based Generators for Edge Intelligence](https://mlsanity.com/paper/1802.09153/) (2018)

## Built on by (most cited first, 11 of 11)

- [ORANSight-2.0: Foundational LLMs for O-RAN](https://mlsanity.com/paper/2503.05200/) (2025)
- [Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review](https://mlsanity.com/paper/2311.11883/) (2023)
- [QEFT: Quantization for Efficient Fine-Tuning of LLMs](https://mlsanity.com/paper/2410.08661/) (2024)
- [Task Vector Quantization for Memory-Efficient Model Merging](https://mlsanity.com/paper/2503.06921/) (2025)
- [Less Precise Can Be More Reliable: A Systematic Evaluation of Quantization's Impact on VLMs Beyond Accuracy](https://mlsanity.com/paper/2509.21173/) (2025)
- [What Survives When You Compress a Recursive Reasoner for the Edge?](https://mlsanity.com/paper/2606.26488/) (2026)
- [How Robustly do LLMs Understand Execution Semantics?](https://mlsanity.com/paper/2604.16320/) (2026)
- [Towards Federated Learning with On-device Training and Communication in 8-bit Floating Point](https://mlsanity.com/paper/2407.02610/) (2024)
- [FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantization](https://mlsanity.com/paper/2506.23516/) (2025)
- [Efficient Deep Neural Receiver with Post-Training Quantization](https://mlsanity.com/paper/2508.06275/) (2025)
- [Algebraic Integer Quantization for Machine Learning Applications](https://mlsanity.com/paper/2610.12494/) (2026)

## Abstract (arXiv)

In this paper we study the effects of quantization in DNN training. We hypothesize that weight quantization is a form of regularization and the amount of regularization is correlated with the quantization level (precision). We confirm our hypothesis by providing analytical study and empirical results. By modeling weight quantization as a form of additive noise to weights, we explore how this noise propagates through the network at training time. We then show that the magnitude of this noise is correlated with the level of quantization. To confirm our analytical study, we performed an extensive list of experiments summarized in this paper in which we show that the regularization effects of quantization can be seen in various vision tasks and models, over various datasets. Based on our study, we propose that 8-bit quantization provides a reliable form of regularization in different vision tasks and models.