mlsanityData to 7 Oct 2026

QReg: On Regularization Effects of Quantization

arXiv 2206.12372 ↗First posted 24 Jun 2022cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper we study the effects of quantization in DNN training. We hypothesize that weight quantization is a form of regularization and the amount of regularization is correlated with the quantization level (precision). We confirm our hypothesis by providing analytical study and empirical results. By modeling weight quantization as a form of additive noise to weights, we explore how this noise propagates through the network at training time. We then show that the magnitude of this noise is correlated with the level of quantization. To confirm our analytical study, we performed an extensive list of experiments summarized in this paper in which we show that the regularization effects of quantization can be seen in various vision tasks and models, over various datasets. Based on our study, we propose that 8-bit quantization provides a reliable form of regularization in different vision tasks and models.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 16 of its 23 references that are in the map. First posted on arXiv 24 Jun 2022.

Builds on

16 of 23 references.

References · 16

  1. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  2. 2019Learned Step Size Quantizationcited by 604
  3. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  4. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  5. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  6. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  7. 2019Towards Universal Object Detection by Domain Attentioncited by 89
  8. 2016Loss-aware Binarization of Deep Networkscited by 56
  9. 2018Loss-aware Weight Quantization of Deep Networkscited by 31
  10. 2018Alternating Multi-bit Quantization for Recurrent Neural Networkscited by 30
  11. 2021Pareto-Optimal Quantized ResNet Is Mostly 4-bitcited by 26
  12. 2018Quantization of Fully Convolutional Networks for Accurate Biomedical Image Segmentationcited by 19
4 more
  1. 2016Deep neural networks are robust to weight binarization and other non-linear distortionscited by 13
  2. 2021Quantization of Deep Neural Networks for Accurate Edge Computingcited by 13
  3. 2022Why Quantization Improves Generalization: NTK of Binary Weight Neural Networkscited by 5
  4. 2018PBGen: Partial Binarization of Deconvolution-Based Generators for Edge Intelligencecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 4; 2026: 2.

9 papers.

  1. 2025ORANSight-2.0: Foundational LLMs for O-RANcited by 13
  2. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  3. 2024QEFT: Quantization for Efficient Fine-Tuning of LLMscited by 7
  4. 2025Task Vector Quantization for Memory-Efficient Model Mergingcited by 5
  5. 2025Less Precise Can Be More Reliable: A Systematic Evaluation of Quantization's Impact on VLMs Beyond Accuracycited by 3
  6. 2026What Survives When You Compress a Recursive Reasoner for the Edge?cited by 3
  7. 2026How Robustly do LLMs Understand Execution Semantics?cited by 2
  8. 2024Towards Federated Learning with On-device Training and Communication in 8-bit Floating Pointcited by 1
  9. 2025FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantizationcited by 1