mlsanityData to 7 Oct 2026

Convolutional Neural Networks Quantization with Attention

arXiv 2209.15317 ↗First posted 30 Sep 2022cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

It has been proven that, compared to using 32-bit floating-point numbers in the training phase, Deep Convolutional Neural Networks (DCNNs) can operate with low precision during inference, thereby saving memory space and power consumption. However, quantizing networks is always accompanied by an accuracy decrease. Here, we propose a method, double-stage Squeeze-and-Threshold (double-stage ST). It uses the attention mechanism to quantize networks and achieve state-of-art results. Using our method, the 3-bit model can achieve accuracy that exceeds the accuracy of the full-precision baseline model. The proposed double-stage ST activation quantization is easy to apply: inserting it before the convolution.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 38 references that are in the map. First posted on arXiv 30 Sep 2022.

Builds on

22 of 38 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  5. 2019Searching for MobileNetV3cited by 1,984
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  7. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  8. 2019Learned Step Size Quantizationcited by 604
  9. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  10. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  11. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  12. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
10 more
  1. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 183
  2. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  3. 2017Deep Learning with Low Precision by Half-wave Gaussian Quantizationcited by 115
  4. 2019XNOR-Net++: Improved Binary Neural Networkscited by 94
  5. 2020HAWQV3: Dyadic Neural Network Quantizationcited by 59
  6. 2021How Do Adam and Training Strategies Help BNNs Optimization?cited by 46
  7. 2018Bi-Real Net: Binarizing Deep Network Towards Real-Network Performancecited by 42
  8. 2019Training Quantized Neural Networks with a Full-precision Auxiliary Modulecited by 35
  9. 2018NICE: Noise Injection and Clamping Estimation for Neural Network Quantizationcited by 24
  10. 2021Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolutioncited by 19

Built on by

Not cited yet.