# Benchmarking Ultra-Low-Power μNPUs

arXiv: https://arxiv.org/abs/2503.22567 · HTML: https://mlsanity.com/paper/2503.22567/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 54 references that are in the map.
- First posted on arXiv 28 Mar 2025.

## Builds on (most cited first, 19 of 19)

- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [MCUNet: Tiny Deep Learning on IoT Devices](https://mlsanity.com/paper/2007.10319/) (2020)
- [On-Device Training Under 256KB Memory](https://mlsanity.com/paper/2206.15472/) (2022)
- [Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions](https://mlsanity.com/paper/2106.05022/) (2021)
- [MELTing point: Mobile Evaluation of Language Transformers](https://mlsanity.com/paper/2403.12844/) (2024)
- [XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inference](https://mlsanity.com/paper/1807.03010/) (2018)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [Efficient Algorithms for Device Placement of DNN Graph Operators](https://mlsanity.com/paper/2006.16423/) (2020)
- [TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge](https://mlsanity.com/paper/2307.09988/) (2023)
- [Reusing Pretrained Models by Multi-linear Operators for Efficient Training](https://mlsanity.com/paper/2310.10699/) (2023)
- [MLPerf Mobile Inference Benchmark](https://mlsanity.com/paper/2012.02328/) (2020)
- [A review of on-device fully neural end-to-end automatic speech recognition algorithms](https://mlsanity.com/paper/2012.07974/) (2020)
- [Flexible and Fully Quantized Ultra-Lightweight TinyissimoYOLO for Ultra-Low-Power Edge Systems](https://mlsanity.com/paper/2307.05999/) (2023)
- [Towards Battery-Free Machine Learning and Inference in Underwater Environments](https://mlsanity.com/paper/2202.08174/) (2022)
- [Complexity-Driven CNN Compression for Resource-constrained Edge AI](https://mlsanity.com/paper/2208.12816/) (2022)
- [BED: A Real-Time Object Detection System for Edge Devices](https://mlsanity.com/paper/2202.07503/) (2022)
- [The Security and Privacy of Mobile Edge Computing: An Artificial Intelligence Perspective](https://mlsanity.com/paper/2401.01589/) (2024)
- [Low-Energy On-Device Personalization for MCUs](https://mlsanity.com/paper/2403.08040/) (2024)
- [DEX: Data Channel Extension for Efficient CNN Inference on Tiny AI Accelerators](https://mlsanity.com/paper/2412.06566/) (2024)

## Built on by (most cited first, 6 of 6)

- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [Proxy Target: Bridging the Gap Between Discrete Spiking Neural Networks and Continuous Control](https://mlsanity.com/paper/2505.24161/) (2025)
- [Performance Analysis of Edge and In-Sensor AI Processors: A Comparative Review](https://mlsanity.com/paper/2603.08725/) (2026)
- [Evaluating the Energy Efficiency of NPU-Accelerated Machine Learning Inference on Embedded Microcontrollers](https://mlsanity.com/paper/2509.17533/) (2025)
- [Energy Constrained Hierarchical Underwater Monitoring via Local Multi-Agent RAG](https://mlsanity.com/paper/2607.24313/) (2026)
- [PolyChirp: Multi-Species Birdsong Classification Using TinyML on Low-Power Acoustic Sensors](https://mlsanity.com/paper/2608.23101/) (2026)

## Abstract (arXiv)

Efficient on-device neural network (NN) inference offers predictable latency, improved privacy and reliability, and lower operating costs for vendors than cloud-based inference. This has sparked recent development of microcontroller-scale NN accelerators, also known as neural processing units (μNPUs), designed specifically for ultra-low-power applications. We present the first comparative evaluation of a number of commercially-available μNPUs, including the first independent benchmarks for multiple platforms. To ensure fairness, we develop and open-source a model compilation pipeline supporting consistent benchmarking of quantized models across diverse microcontroller hardware. Our resulting analysis uncovers both expected performance trends as well as surprising disparities between hardware specifications and actual performance, including certain μNPUs exhibiting unexpected scaling behaviors with model complexity. This work provides a foundation for ongoing evaluation of μNPU platforms, alongside offering practical insights for both hardware and software developers in this rapidly evolving space.