mlsanityData to 7 Oct 2026

n-hot: Efficient bit-level sparsity for powers-of-two neural network quantization

arXiv 2103.11704 ↗First posted 22 Mar 2021cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Powers-of-two (PoT) quantization reduces the number of bit operations of deep neural networks on resource-constrained hardware. However, PoT quantization triggers a severe accuracy drop because of its limited representation ability. Since DNN models have been applied for relatively complex tasks (e.g., classification for large datasets and object detection), improvement in accuracy for the PoT quantization method is required. Although some previous works attempt to improve the accuracy of PoT quantization, there is no work that balances accuracy and computation costs in a memory-efficient way. To address this problem, we propose an efficient PoT quantization scheme. Bit-level sparsity is introduced; weights (or activations) are rounded to values that can be calculated by n shift operations in multiplication. We also allow not only addition but also subtraction as each operation. Moreover, we use a two-stage fine-tuning algorithm to recover the accuracy drop that is triggered by introducing the bit-level sparsity. The experimental results on an object detection model (CenterNet, MobileNet-v2 backbone) on the COCO dataset show that our proposed method suppresses the accuracy drop by 0.3% at most while reducing the number of operations by about 75% and model size by 11.5% compared to the uniform method.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 47 references that are in the map. First posted on arXiv 22 Mar 2021.

Builds on

34 of 47 references.

References · 34

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  6. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  7. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  8. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  9. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  10. 2019Objects as Pointscited by 1,172
  11. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  12. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
22 more
  1. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  2. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  3. 2019Importance Estimation for Neural Network Pruningcited by 531
  4. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  5. 2018Model compression via distillation and quantizationcited by 287
  6. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  7. 2013Predicting Parameters in Deep Learningcited by 223
  8. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  9. 2016Ternary Weight Networkscited by 195
  10. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
  11. 2015Fixed Point Quantization of Deep Convolutional Networkscited by 141
  12. 2018Scalable Methods for 8-bit Training of Neural Networkscited by 139
  13. 2016Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1cited by 102
  14. 2016Convolutional Neural Networks using Logarithmic Data Representationcited by 97
  15. 2019AdderNet: Do We Really Need Multiplications in Deep Learning?cited by 82
  16. 2017Towards Effective Low-bitwidth Convolutional Neural Networkscited by 76
  17. 2019Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Modelcited by 58
  18. 2018Attention-Aware Compositional Network for Person Re-identificationcited by 50
  19. 2018Deep Object-Centric Policies for Autonomous Drivingcited by 41
  20. 2018Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentationcited by 37
  21. 2021CPT: Efficient Deep Neural Network Training via Cyclic Precisioncited by 19
  22. 2016Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detectioncited by 8

Built on by

Not cited yet.