# Quark: An Integer RISC-V Vector Processor for Sub-Byte Quantized DNN Inference

arXiv: https://arxiv.org/abs/2302.05996 · HTML: https://mlsanity.com/paper/2302.05996/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 12 Feb 2023.

## Builds on (most cited first, 13 of 13)

- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [FINN: A Framework for Fast, Scalable Binarized Neural Network Inference](https://mlsanity.com/paper/1612.07119/) (2016)
- [Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks](https://mlsanity.com/paper/1712.01507/) (2017)
- [ReCU: Reviving the Dead Weights in Binary Neural Networks](https://mlsanity.com/paper/2103.12369/) (2021)
- [A near-threshold RISC-V core with DSP extensions for scalable IoT Endpoint Devices](https://mlsanity.com/paper/1608.08376/) (2016)
- [The Cost of Application-Class Processing: Energy and Performance Analysis of a Linux-ready 1.7GHz 64bit RISC-V Core in 22nm FDSOI Technology](https://mlsanity.com/paper/1904.05442/) (2019)
- [A Mixed-Precision RISC-V Processor for Extreme-Edge DNN Inference](https://mlsanity.com/paper/2010.04073/) (2020)
- [Loom: Exploiting Weight and Activation Precisions to Accelerate Convolutional Neural Networks](https://mlsanity.com/paper/1706.07853/) (2017)
- [BARVINN: Arbitrary Precision DNN Accelerator Controlled by a RISC-V CPU](https://mlsanity.com/paper/2301.00290/) (2022)
- [Ara: A 1 GHz+ Scalable and Energy-Efficient RISC-V Vector Processor with Multi-Precision Floating Point Support in 22 nm FD-SOI](https://mlsanity.com/paper/1906.00478/) (2019)
- [A "New Ara" for Vector Computing: An Open Source Highly Efficient RISC-V V 1.0 Vector Processor Design](https://mlsanity.com/paper/2210.08882/) (2022)

## Built on by (most cited first, 2 of 2)

- [SPEED: A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference](https://mlsanity.com/paper/2409.14017/) (2024)
- [Sparq: A Custom RISC-V Vector Processor for Efficient Sub-Byte Quantized Inference](https://mlsanity.com/paper/2306.09905/) (2023)

## Abstract (arXiv)

In this paper, we present Quark, an integer RISC-V vector processor specifically tailored for sub-byte DNN inference. Quark is implemented in GlobalFoundries' 22FDX FD-SOI technology. It is designed on top of Ara, an open-source 64-bit RISC-V vector processor. To accommodate sub-byte DNN inference, Quark extends Ara by adding specialized vector instructions to perform sub-byte quantized operations. We also remove the floating-point unit from Quarks' lanes and use the CVA6 RISC-V scalar core for the re-scaling operations that are required in quantized neural network inference. This makes each lane of Quark 2 times smaller and 1.9 times more power efficient compared to the ones of Ara. In this paper we show that Quark can run quantized models at sub-byte precision. Notably we show that for 1-bit and 2-bit quantized models, Quark can accelerate computation of Conv2d over various ranges of inputs and kernel sizes.