# PULP-NN: Accelerating Quantized Neural Networks on Parallel Ultra-Low-Power RISC-V Processors

arXiv: https://arxiv.org/abs/1908.11263 · HTML: https://mlsanity.com/paper/1908.11263/

## Key facts

- Cited by 34 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 29 Aug 2019.

## Built on by (most cited first, 30 of 34)

- [Auto-Split: A General Framework of Collaborative Edge-Cloud AI](https://mlsanity.com/paper/2108.13041/) (2021)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [A Machine Learning-oriented Survey on Tiny Machine Learning](https://mlsanity.com/paper/2309.11932/) (2023)
- [PhiNets: a scalable backbone for low-power AI at the edge](https://mlsanity.com/paper/2110.00337/) (2021)
- [MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems from Microwatts to Megawatts for Sustainable AI](https://mlsanity.com/paper/2410.12032/) (2024)
- [ECG-TCN: Wearable Cardiac Arrhythmia Detection with a Temporal Convolutional Network](https://mlsanity.com/paper/2103.13740/) (2021)
- [Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design](https://mlsanity.com/paper/2103.15750/) (2021)
- [Sound Event Detection with Binary Neural Networks on Tightly Power-Constrained IoT Devices](https://mlsanity.com/paper/2101.04446/) (2021)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [Optimizing the Deployment of Tiny Transformers on Low-Power MCUs](https://mlsanity.com/paper/2404.02945/) (2024)
- [Ultra-compact Binary Neural Networks for Human Activity Recognition on RISC-V Processors](https://mlsanity.com/paper/2205.12781/) (2022)
- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [MATCH: Model-Aware TVM-based Compilation for Heterogeneous Edge Devices](https://mlsanity.com/paper/2410.08855/) (2024)
- [TCN Mapping Optimization for Ultra-Low Power Time-Series Edge Inference](https://mlsanity.com/paper/2203.12925/) (2022)
- [xTern: Energy-Efficient Ternary Neural Network Inference on RISC-V-Based Edge Systems](https://mlsanity.com/paper/2405.19065/) (2024)
- [Energy-Efficient Adaptive Machine Learning on IoT End-Nodes With Class-Dependent Confidence](https://mlsanity.com/paper/2204.03431/) (2022)
- [A Map-free Deep Learning-based Framework for Gate-to-Gate Monocular Visual Navigation aboard Miniaturized Aerial Vehicles](https://mlsanity.com/paper/2503.05251/) (2025)
- [Shifting Capsule Networks from the Cloud to the Deep Edge](https://mlsanity.com/paper/2110.02911/) (2021)
- [HiKonv: High Throughput Quantized Convolution With Novel Bit-wise Management and Computation](https://mlsanity.com/paper/2112.13972/) (2021)
- [A Heterogeneous In-Memory Computing Cluster For Flexible End-to-End Inference of Real-World Deep Neural Networks](https://mlsanity.com/paper/2201.01089/) (2022)
- [Optimization and Deployment of Deep Neural Networks for PPG-based Blood Pressure Estimation Targeting Low-power Wearables](https://mlsanity.com/paper/2409.07485/) (2024)
- [Lightweight Software Kernels and Hardware Extensions for Efficient Sparse Deep Neural Networks on Microcontrollers](https://mlsanity.com/paper/2503.06183/) (2025)
- [Compiler Toolchains for Deep Learning Workloads on Embedded Platforms](https://mlsanity.com/paper/2104.04576/) (2021)
- [Energy-frugal and Interpretable AI Hardware Design using Learning Automata](https://mlsanity.com/paper/2305.11928/) (2023)
- [Decentor-V: Lightweight ML Training on Low-Power RISC-V Edge Devices](https://mlsanity.com/paper/2509.18118/) (2025)
- [FEMBA on the Edge: Physiologically-Aware Pre-Training, Quantization, and Deployment of a Bidirectional Mamba EEG Foundation Model on an Ultra-low Power Microcontroller](https://mlsanity.com/paper/2603.26716/) (2026)
- [Compilation and Optimizations for Efficient Machine Learning on Embedded Systems](https://mlsanity.com/paper/2206.03326/) (2022)
- [HiKonv: Maximizing the Throughput of Quantized Convolution With Novel Bit-wise Management and Computation](https://mlsanity.com/paper/2208.00763/) (2022)
- [Accelerating Depthwise Separable Convolutions on Ultra-Low-Power Devices](https://mlsanity.com/paper/2406.12478/) (2024)
- [Optimizing DNN Inference on Multi-Accelerator SoCs at Training-time](https://mlsanity.com/paper/2409.18566/) (2024)

## Abstract (arXiv)

We present PULP-NN, an optimized computing library for a parallel ultra-low-power tightly coupled cluster of RISC-V processors. The key innovation in PULP-NN is a set of kernels for Quantized Neural Network (QNN) inference, targeting byte and sub-byte data types, down to INT-1, tuned for the recent trend toward aggressive quantization in deep neural network inference. The proposed library exploits both the digital signal processing (DSP) extensions available in the PULP RISC-V processors and the cluster's parallelism, achieving up to 15.5 MACs/cycle on INT-8 and improving performance by up to 63x with respect to a sequential implementation on a single RISC-V core implementing the baseline RV32IMC ISA. Using PULP-NN, a CIFAR-10 network on an octa-core cluster runs in 30x and 19.6x less clock cycles than the current state-of-the-art ARM CMSIS-NN library, running on STM32L4 and STM32H7 MCUs, respectively. The proposed library, when running on GAP-8 processor, outperforms by 36.8x and by 7.45x the execution on energy efficient MCUs such as STM32L4 and high-end MCUs such as STM32H7 respectively, when operating at the maximum frequency. The energy efficiency on GAP-8 is 14.1x higher than STM32L4 and 39.5x higher than STM32H7, at the maximum efficiency operating point.