# Accelerating Deep Convolutional Networks using low-precision and sparsity

arXiv: https://arxiv.org/abs/1610.00324 · HTML: https://mlsanity.com/paper/1610.00324/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2016.

## Built on by (most cited first, 8 of 8)

- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [Geometry-aware training of factorized layers in tensor Tucker format](https://mlsanity.com/paper/2305.19059/) (2023)
- [Quantum Denoising Diffusion Models](https://mlsanity.com/paper/2401.07049/) (2024)
- [Rethinking Floating Point Overheads for Mixed Precision DNN Accelerators](https://mlsanity.com/paper/2101.11748/) (2021)
- [TinyMLOps: Operational Challenges for Widespread Edge AI Adoption](https://mlsanity.com/paper/2203.10923/) (2022)
- [Xtreme Margin: A Tunable Loss Function for Binary Classification Problems](https://mlsanity.com/paper/2211.00176/) (2022)
- [Energy awareness in low precision neural networks](https://mlsanity.com/paper/2202.02783/) (2022)

## Abstract (arXiv)

We explore techniques to significantly improve the compute efficiency and performance of Deep Convolution Networks without impacting their accuracy. To improve the compute efficiency, we focus on achieving high accuracy with extremely low-precision (2-bit) weight networks, and to accelerate the execution time, we aggressively skip operations on zero-values. We achieve the highest reported accuracy of 76.6% Top-1/93% Top-5 on the Imagenet object classification challenge with low-precision networkfootnotegithub release of the source code coming soon while reducing the compute requirement by ~3x compared to a full-precision network that achieves similar accuracy. Furthermore, to fully exploit the benefits of our low-precision networks, we build a deep learning accelerator core, dLAC, that can achieve up to 1 TFLOP/mm2 equivalent for single-precision floating-point operations (~2 TFLOP/mm2 for half-precision).