# Benchmarking Energy and Latency in TinyML: A Novel Method for Resource-Constrained AI

arXiv: https://arxiv.org/abs/2505.15622 · HTML: https://mlsanity.com/paper/2505.15622/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 26 references that are in the map.
- First posted on arXiv 21 May 2025.

## Builds on (most cited first, 4 of 4)

- [MLPerf Inference Benchmark](https://mlsanity.com/paper/1911.02549/) (2019)
- [MLPerf Tiny Benchmark](https://mlsanity.com/paper/2106.07597/) (2021)
- [Benchmarking TinyML Systems: Challenges and Direction](https://mlsanity.com/paper/2003.04821/) (2020)
- [MLonMCU: TinyML Benchmarking with Fast Retargeting](https://mlsanity.com/paper/2306.08951/) (2023)

## Built on by (most cited first, 4 of 4)

- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [End-to-End Efficiency in Keyword Spotting: A System-Level Approach for Embedded Microcontrollers](https://mlsanity.com/paper/2509.07051/) (2025)
- [Efficient Sensor Fusion for Gesture Recognition on Resource-Constrained Devices](https://mlsanity.com/paper/2605.13462/) (2026)
- [CREST: Deployment-Realistic Hardware-in-the-Loop NAS for Embedded Sensing Systems](https://mlsanity.com/paper/2606.15004/) (2026)

## Abstract (arXiv)

The rise of IoT has increased the need for on-edge machine learning, with TinyML emerging as a promising solution for resource-constrained devices such as MCU. However, evaluating their performance remains challenging due to diverse architectures and application scenarios. Current solutions have many non-negligible limitations. This work introduces an alternative benchmarking methodology that integrates energy and latency measurements while distinguishing three execution phases pre-inference, inference, and post-inference. Additionally, the setup ensures that the device operates without being powered by an external measurement unit, while automated testing can be leveraged to enhance statistical significance. To evaluate our setup, we tested the STM32N6 MCU, which includes a NPU for executing neural networks. Two configurations were considered: high-performance and Low-power. The variation of the EDP was analyzed separately for each phase, providing insights into the impact of hardware configurations on energy efficiency. Each model was tested 1000 times to ensure statistically relevant results. Our findings demonstrate that reducing the core voltage and clock frequency improve the efficiency of pre- and post-processing without significantly affecting network execution performance. This approach can also be used for cross-platform comparisons to determine the most efficient inference platform and to quantify how pre- and post-processing overhead varies across different hardware implementations.