# HiKonv: High Throughput Quantized Convolution With Novel Bit-wise Management and Computation

arXiv: https://arxiv.org/abs/2112.13972 · HTML: https://mlsanity.com/paper/2112.13972/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 28 Dec 2021.

## Builds on (most cited first, 11 of 11)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs](https://mlsanity.com/paper/1801.06601/) (2018)
- [Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks](https://mlsanity.com/paper/1712.01507/) (2017)
- [FPGA/DNN Co-Design: An Efficient Design Methodology for IoT Intelligence on the Edge](https://mlsanity.com/paper/1904.04421/) (2019)
- [PULP-NN: Accelerating Quantized Neural Networks on Parallel Ultra-Low-Power RISC-V Processors](https://mlsanity.com/paper/1908.11263/) (2019)
- [Survey of Machine Learning Accelerators](https://mlsanity.com/paper/2009.00993/) (2020)
- [SkyNet: a Hardware-Efficient Method for Object Detection and Tracking on Embedded Systems](https://mlsanity.com/paper/1909.09709/) (2019)
- [VecQ: Minimal Loss DNN Model Compression With Vectorized Weight Quantization](https://mlsanity.com/paper/2005.08501/) (2020)
- [A Mixed-Precision RISC-V Processor for Extreme-Edge DNN Inference](https://mlsanity.com/paper/2010.04073/) (2020)
- [Loom: Exploiting Weight and Activation Precisions to Accelerate Convolutional Neural Networks](https://mlsanity.com/paper/1706.07853/) (2017)

## Built on by (most cited first, 6 of 6)

- [Sensitivity-Aware Mixed-Precision Quantization and Width Optimization of Deep Neural Networks Through Cluster-Based Tree-Structured Parzen Estimation](https://mlsanity.com/paper/2308.06422/) (2023)
- [DeepBurning-MixQ: An Open Source Mixed-Precision Neural Network Accelerator Design Framework for FPGAs](https://mlsanity.com/paper/2308.11334/) (2023)
- [TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecture](https://mlsanity.com/paper/2411.03697/) (2024)
- [MCU-MixQ: A HW/SW Co-optimized Mixed-precision Neural Network Design Framework for MCUs](https://mlsanity.com/paper/2407.18267/) (2024)
- [XtraMAC: An Efficient MAC Architecture for Mixed-Precision LLM Inference on FPGA](https://mlsanity.com/paper/2605.06052/) (2026)
- [Arithmetic Packing on Wide Integer Datapaths in DSP Primitives of Modern FPGA Devices](https://mlsanity.com/paper/2606.11065/) (2026)

## Abstract (arXiv)

Quantization for Convolutional Neural Network (CNN) has shown significant progress with the intention of reducing the cost of computation and storage with low-bitwidth data inputs. There are, however, no systematic studies on how an existing full-bitwidth processing unit, such as CPUs and DSPs, can be better utilized to carry out significantly higher computation throughput for convolution under various quantized bitwidths. In this study, we propose HiKonv, a unified solution that maximizes the compute throughput of a given underlying processing unit to process low-bitwidth quantized data inputs through novel bit-wise parallel computation. We establish theoretical performance bounds using a full-bitwidth multiplier for highly parallelized low-bitwidth convolution, and demonstrate new breakthroughs for high-performance computing in this critical domain. For example, a single 32-bit processing unit can deliver 128 binarized convolution operations (multiplications and additions) under one CPU instruction, and a single 27x18 DSP core can deliver eight convolution operations with 4-bit inputs in one cycle. We demonstrate the effectiveness of HiKonv on CPU and FPGA for both convolutional layers or a complete DNN model. For a convolutional layer quantized to 4-bit, HiKonv achieves a 3.17x latency improvement over the baseline implementation using C++ on CPU. Compared to the DAC-SDC 2020 champion model for FPGA, HiKonv achieves a 2.37x throughput improvement and 2.61x DSP efficiency improvement, respectively.