mlsanityData to 14 Oct 2026

Standard Deviation-Based Quantization for Deep Neural Networks

arXiv 2202.12422 ↗First posted 24 Feb 2022cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Quantization of deep neural networks is a promising approach that reduces the inference cost, making it feasible to run deep networks on resource-restricted devices. Inspired by existing methods, we propose a new framework to learn the quantization intervals (discrete values) using the knowledge of the network's weight and activation distributions, i.e., standard deviation. Furthermore, we propose a novel base-2 logarithmic quantization scheme to quantize weights to power-of-two discrete values. Our proposed scheme allows us to replace resource-hungry high-precision multipliers with simple shift-add operations. According to our evaluations, our method outperforms existing work on CIFAR10 and ImageNet datasets and even achieves better accuracy performance with 3-bit weights and activations when compared to the full-precision models. Moreover, our scheme simultaneously prunes the network's parameters and allows us to flexibly adjust the pruning ratio during the quantization process.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 24 Feb 2022.

Builds on

27 of 37 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  7. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  8. 2019Searching for MobileNetV3cited by 1,994
  9. 2019Learned Step Size Quantizationcited by 628
  10. 2018Rethinking the Value of Network Pruningcited by 599
  11. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  12. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
15 more
  1. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  2. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 274
  3. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 238
  4. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
  5. 2016Ternary Weight Networkscited by 204
  6. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 160
  7. 2017Deep Learning with Low Precision by Half-wave Gaussian Quantizationcited by 117
  8. 2016Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1cited by 105
  9. 2016Convolutional Neural Networks using Logarithmic Data Representationcited by 99
  10. 2017Towards Effective Low-bitwidth Convolutional Neural Networkscited by 78
  11. 2018SqueezeNext: Hardware-Aware Neural Network Designcited by 61
  12. 2019DeepShift: Towards Multiplication-Less Neural Networkscited by 54
  13. 2020ShiftAddNet: A Hardware-Inspired Deep Networkcited by 52
  14. 2015VLSI Implementation of Deep Neural Network Using Integral Stochastic Computingcited by 7
  15. 2021S3: Sign-Sparse-Shift Reparametrization for Effective Training of Low-bit Shift Networkscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022A simple approach for quantizing neural networks
  2. 2025Low-bit Model Quantization for Deep Neural Networks: A Survey
  3. 2022A Closer Look at Hardware-Friendly Weight Quantization
  4. 2023Designing strong baselines for ternary neural network quantization through support and mass equalization
  5. 2022Power-of-Two Quantization for Low Bitwidth and Hardware Compliant Neural Networks
  6. 2021Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats
  7. 2023SPFQ: A Stochastic Algorithm and Its Error Analysis for Neural Network Quantization
  8. 2024A Data-Free Analytical Quantization Scheme for Deep Learning Models