# Streamlined Deployment for Quantized Neural Networks

arXiv: https://arxiv.org/abs/1709.04060 · HTML: https://mlsanity.com/paper/1709.04060/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2017.

## Built on by (most cited first, 12 of 12)

- [Applications and Techniques for Fast Machine Learning in Science](https://mlsanity.com/paper/2110.13041/) (2021)
- [A2Q: Accumulator-Aware Quantization with Guaranteed Overflow Avoidance](https://mlsanity.com/paper/2308.13504/) (2023)
- [Open-source FPGA-ML codesign for the MLPerf Tiny Benchmark](https://mlsanity.com/paper/2206.11791/) (2022)
- [A2Q+: Improving Accumulator-Aware Weight Quantization](https://mlsanity.com/paper/2401.10432/) (2024)
- [Benchmarking Quantized Neural Networks on FPGAs with FINN](https://mlsanity.com/paper/2102.01341/) (2021)
- [Sensitivity-Guided Framework for Pruned and Quantized Reservoir Computing Accelerators](https://mlsanity.com/paper/2603.08737/) (2026)
- [Mitigating scalability challenges in LUT-based neural networks via pruning optimisations](https://mlsanity.com/paper/2407.02362/) (2024)
- [Real-Time Multi-Object Tracking using YOLOv8 and SORT on a SoC FPGA](https://mlsanity.com/paper/2503.13023/) (2025)
- [FINN-GL: Generalized Mixed-Precision Extensions for FPGA-Accelerated LSTMs](https://mlsanity.com/paper/2506.20810/) (2025)
- [Red grape detection with accelerated artificial neural networks in the FPGA's programmable logic](https://mlsanity.com/paper/2507.02443/) (2025)
- [SIRA: Scaled-Integer Range Analysis for Optimizing FPGA Dataflow Neural Network Accelerators](https://mlsanity.com/paper/2508.21493/) (2025)
- [Deep Defence on Wheels: A Dual Intrusion Detection System Architecture for Comprehensive In-Vehicle Network Security](https://mlsanity.com/paper/2610.07489/) (2026)

## Abstract (arXiv)

Running Deep Neural Network (DNN) models on devices with limited computational capability is a challenge due to large compute and memory requirements. Quantized Neural Networks (QNNs) have emerged as a potential solution to this problem, promising to offer most of the DNN accuracy benefits with much lower computational cost. However, harvesting these benefits on existing mobile CPUs is a challenge since operations on highly quantized datatypes are not natively supported in most instruction set architectures (ISAs). In this work, we first describe a streamlining flow to convert all QNN inference operations to integer ones. Afterwards, we provide techniques based on processing one bit position at a time (bit-serial) to show how QNNs can be efficiently deployed using common bitwise operations. We demonstrate the potential of QNNs on mobile CPUs with microbenchmarks and on a quantized AlexNet, which is 3.5x faster than an optimized 8-bit baseline. Our bit-serial matrix multiplication library is available on GitHub at https://git.io/vhshn