# Fast Implementation of 4-bit Convolutional Neural Networks for Mobile Devices

arXiv: https://arxiv.org/abs/2009.06488 · HTML: https://mlsanity.com/paper/2009.06488/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2020.

## Built on by (most cited first, 3 of 3)

- [Fast matrix multiplication for binary and ternary CNNs on ARM CPU](https://mlsanity.com/paper/2205.09120/) (2022)
- [Measurement-driven neural-network training for integrated magnetic tunnel junction arrays](https://mlsanity.com/paper/2312.06446/) (2023)
- [Implementation of a Binary Neural Network on a Passive Array of Magnetic Tunnel Junctions](https://mlsanity.com/paper/2112.09159/) (2021)

## Abstract (arXiv)

Quantized low-precision neural networks are very popular because they require less computational resources for inference and can provide high performance, which is vital for real-time and embedded recognition systems. However, their advantages are apparent for FPGA and ASIC devices, while general-purpose processor architectures are not always able to perform low-bit integer computations efficiently. The most frequently used low-precision neural network model for mobile central processors is an 8-bit quantized network. However, in a number of cases, it is possible to use fewer bits for weights and activations, and the only problem is the difficulty of efficient implementation. We introduce an efficient implementation of 4-bit matrix multiplication for quantized neural networks and perform time measurements on a mobile ARM processor. It shows 2.9 times speedup compared to standard floating-point multiplication and is 1.5 times faster than 8-bit quantized one. We also demonstrate a 4-bit quantized neural network for OCR recognition on the MIDV-500 dataset. 4-bit quantization gives 95.0% accuracy and 48% overall inference speedup, while an 8-bit quantized network gives 95.4% accuracy and 39% speedup. The results show that 4-bit quantization perfectly suits mobile devices, yielding good enough accuracy and low inference time.