A 3 TOPS/W RISC-V Parallel Cluster for Inference of Fine-Grain Mixed-Precision Quantized Neural Networks
Not on the map. Why?
Abstract (arXiv)
The emerging trend of deploying complex algorithms, such as Deep Neural Networks (DNNs), increasingly poses strict memory and energy efficiency requirements on Internet-of-Things (IoT) end-nodes. Mixed-precision quantization has been proposed as a technique to minimize a DNN's memory footprint and maximize its execution efficiency, with negligible end-to-end precision degradation. In this work, we present a novel hardware and software stack for energy-efficient inference of mixed-precision Quantized Neural Networks (QNNs). We introduce Flex-V, a processor based on the RISC-V Instruction Set Architecture (ISA) that features fused Mac&Load mixed-precision dot product instructions; to avoid the exponential growth of the encoding space due to mixed-precision variants, we encode formats into the Control-Status Registers (CSRs). Flex-V core is integrated into a tightly-coupled cluster of eight processors; in addition, we provide a full framework for the end-to-end deployment of DNNs including a compiler, optimized libraries, and a memory-aware deployment flow. Our results show up to 91.5 MAC/cycle and 3.26 TOPS/W on the cluster, implemented in a commercial 22nm FDX technology, with up to 8.5x speed-up, and an area overhead of only 5.6% with respect to the baseline. To demonstrate the capabilities of the architecture, we benchmark it with end-to-end real-life QNNs, improving performance by 2x - 2.5x with respect to existing solutions using fully flexible programmable processors.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 11 of its 21 references that are in the map. First posted on arXiv 3 Jul 2023.
Builds on
11 of 21 references.
References · 11
- 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
- 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
- 2016FINN: A Framework for Fast, Scalable Binarized Neural Network Inferencecited by 150
- 2018CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUscited by 121
- 2020DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUscited by 58
- 2019Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollerscited by 33
- 2021Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Modecited by 18
- 2016A near-threshold RISC-V core with DSP extensions for scalable IoT Endpoint Devicescited by 17
- 2020A Mixed-Precision RISC-V Processor for Extreme-Edge DNN Inferencecited by 13
- 2020Enabling Mixed-Precision Quantized Neural Networks in Extreme-Edge Devicescited by 10
- 2020XpulpNN: Enabling Energy Efficient and Flexible Inference of Quantized Neural Network on RISC-V based IoT End Nodescited by 7
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2024Mixed-precision Neural Networks on RISC-V Cores: ISA extensions for Multi-Pumped Soft SIMD Operations
- 2025MaRVIn: A Cross-Layer Mixed-Precision RISC-V Framework for DNN Inference, from ISA Extension to Hardware Acceleration
- 2023Free Bits: Latency Optimization of Mixed-Precision Quantized Neural Networks on the Edge
- 2023A Precision-Scalable RISC-V DNN Processor with On-Device Learning Capability at the Extreme Edge
- 2023On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networks
- 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications
- 2022Channel-wise Mixed-precision Assignment for DNN Inference on Constrained Edge Nodes
- 2023A Practical Mixed Precision Algorithm for Post-Training Quantization