# Elastic Significant Bit Quantization and Acceleration for Deep Neural Networks

arXiv: https://arxiv.org/abs/2109.03513 · HTML: https://mlsanity.com/paper/2109.03513/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 48 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [Deeply-Supervised Nets](https://mlsanity.com/paper/1409.5185/) (2014)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks](https://mlsanity.com/paper/1908.05033/) (2019)
- [Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights](https://mlsanity.com/paper/1702.03044/) (2017)
- [Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss](https://mlsanity.com/paper/1808.05779/) (2018)
- [Compressing Neural Networks with the Hashing Trick](https://mlsanity.com/paper/1504.04788/) (2015)
- [Towards Accurate Binary Convolutional Neural Network](https://mlsanity.com/paper/1711.11294/) (2017)
- [Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks](https://mlsanity.com/paper/1909.13144/) (2019)
- [Convolutional Neural Networks using Logarithmic Data Representation](https://mlsanity.com/paper/1603.01025/) (2016)
- [PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning](https://mlsanity.com/paper/2001.00138/) (2020)
- [AutoQ: Automated Kernel-Wise Neural Network Quantization](https://mlsanity.com/paper/1902.05690/) (2019)
- [Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMM](https://mlsanity.com/paper/1707.09870/) (2017)
- [Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks](https://mlsanity.com/paper/1711.02213/) (2017)
- [SYQ: Learning Symmetric Quantization For Efficient Deep Neural Networks](https://mlsanity.com/paper/1807.00301/) (2018)
- [Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networks](https://mlsanity.com/paper/1808.05240/) (2018)
- [VecQ: Minimal Loss DNN Model Compression With Vectorized Weight Quantization](https://mlsanity.com/paper/2005.08501/) (2020)
- [Deep Positron: A Deep Neural Network Using the Posit Number System](https://mlsanity.com/paper/1812.01762/) (2018)
- [Design Flow of Accelerating Hybrid Extremely Low Bit-width Neural Network in Embedded FPGA](https://mlsanity.com/paper/1808.04311/) (2018)
- [AdaptivFloat: A Floating-point based Data Type for Resilient Deep Learning Inference](https://mlsanity.com/paper/1909.13271/) (2019)
- [AddNet: Deep Neural Networks Using FPGA-Optimized Multipliers](https://mlsanity.com/paper/1911.08097/) (2019)

## Built on by (most cited first, 4 of 4)

- [Deep Feature Surgery: Towards Accurate and Efficient Multi-Exit Networks](https://mlsanity.com/paper/2407.13986/) (2024)
- [Minimize Quantization Output Error with Bias Compensation](https://mlsanity.com/paper/2404.01892/) (2024)
- [Accelerating PoT Quantization on Edge Devices](https://mlsanity.com/paper/2409.20403/) (2024)
- [PoTAcc: A Pipeline for End-to-End Acceleration of Power-of-Two Quantized DNNs](https://mlsanity.com/paper/2605.06082/) (2026)

## Abstract (arXiv)

Quantization has been proven to be a vital method for improving the inference efficiency of deep neural networks (DNNs). However, it is still challenging to strike a good balance between accuracy and efficiency while quantizing DNN weights or activation values from high-precision formats to their quantized counterparts. We propose a new method called elastic significant bit quantization (ESB) that controls the number of significant bits of quantized values to obtain better inference accuracy with fewer resources. We design a unified mathematical formula to constrain the quantized values of the ESB with a flexible number of significant bits. We also introduce a distribution difference aligner (DDA) to quantitatively align the distributions between the full-precision weight or activation values and quantized values. Consequently, ESB is suitable for various bell-shaped distributions of weights and activation of DNNs, thus maintaining a high inference accuracy. Benefitting from fewer significant bits of quantized values, ESB can reduce the multiplication complexity. We implement ESB as an accelerator and quantitatively evaluate its efficiency on FPGAs. Extensive experimental results illustrate that ESB quantization consistently outperforms state-of-the-art methods and achieves average accuracy improvements of 4.78%, 1.92%, and 3.56% over AlexNet, ResNet18, and MobileNetV2, respectively. Furthermore, ESB as an accelerator can achieve 10.95 GOPS peak performance of 1k LUTs without DSPs on the Xilinx ZCU102 FPGA platform. Compared with CPU, GPU, and state-of-the-art accelerators on FPGAs, the ESB accelerator can improve the energy efficiency by up to 65x, 11x, and 26x, respectively.