# Evaluating the Energy Efficiency of NPU-Accelerated Machine Learning Inference on Embedded Microcontrollers

arXiv: https://arxiv.org/abs/2509.17533 · HTML: https://mlsanity.com/paper/2509.17533/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 10 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Builds on (most cited first, 7 of 7)

- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [Structured Pruning of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1512.08571/) (2015)
- [Benchmarking TinyML Systems: Challenges and Direction](https://mlsanity.com/paper/2003.04821/) (2020)
- [FlexNN: A Dataflow-aware Flexible Deep Learning Accelerator for Energy-Efficient Edge Devices](https://mlsanity.com/paper/2403.09026/) (2024)
- [Benchmarking Ultra-Low-Power μNPUs](https://mlsanity.com/paper/2503.22567/) (2025)
- [dCSR: A Memory-Efficient Sparse Matrix Representation for Parallel Neural Network Inference](https://mlsanity.com/paper/2111.12345/) (2021)

## Abstract (arXiv)

The deployment of machine learning (ML) models on microcontrollers (MCUs) is constrained by strict energy, latency, and memory requirements, particularly in battery-operated and real-time edge devices. While software-level optimizations such as quantization and pruning reduce model size and computation, hardware acceleration has emerged as a decisive enabler for efficient embedded inference. This paper evaluates the impact of Neural Processing Units (NPUs) on MCU-based ML execution, using the ARM Cortex-M55 core combined with the Ethos-U55 NPU on the Alif Semiconductor Ensemble E7 development board as a representative platform. A rigorous measurement methodology was employed, incorporating per-inference net energy accounting via GPIO-triggered high-resolution digital multimeter synchronization and idle-state subtraction, ensuring accurate attribution of energy costs. Experimental results across six representative ML models -including MiniResNet, MobileNetV2, FD-MobileNet, MNIST, TinyYolo, and SSD-MobileNet-demonstrate substantial efficiency gains when inference is offloaded to the NPU. For moderate to large networks, latency improvements ranged from 7x to over 125x, with per-inference net energy reductions up to 143x. Notably, the NPU enabled execution of models unsupported on CPU-only paths, such as SSD-MobileNet, highlighting its functional as well as efficiency advantages. These findings establish NPUs as a cornerstone of energy-aware embedded AI, enabling real-time, power-constrained ML inference at the MCU level.