# FANN-on-MCU: An Open-Source Toolkit for Energy-Efficient Neural Network Inference at the Edge of the Internet of Things

arXiv: https://arxiv.org/abs/1911.03314 · HTML: https://mlsanity.com/paper/1911.03314/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2019.

## Built on by (most cited first, 18 of 18)

- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [Quantization and Deployment of Deep Neural Networks on Microcontrollers](https://mlsanity.com/paper/2105.13331/) (2021)
- [TinyissimoYOLO: A Quantized, Low-Memory Footprint, TinyML Object Detection Network for Low Power Microcontrollers](https://mlsanity.com/paper/2306.00001/) (2023)
- [Hypervector Design for Efficient Hyperdimensional Computing on Edge Devices](https://mlsanity.com/paper/2103.06709/) (2021)
- [TinyML: Tools, Applications, Challenges, and Future Research Directions](https://mlsanity.com/paper/2303.13569/) (2023)
- [Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design](https://mlsanity.com/paper/2103.15750/) (2021)
- [Sound Event Detection with Binary Neural Networks on Tightly Power-Constrained IoT Devices](https://mlsanity.com/paper/2101.04446/) (2021)
- [Dynamic ConvNets on Tiny Devices via Nested Sparsity](https://mlsanity.com/paper/2203.03324/) (2022)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [PicoSAM2: Low-Latency Segmentation In-Sensor for Edge Vision Applications](https://mlsanity.com/paper/2506.18807/) (2025)
- [TCN Mapping Optimization for Ultra-Low Power Time-Series Edge Inference](https://mlsanity.com/paper/2203.12925/) (2022)
- [An Open-Source Tool for Classification Models in Resource-Constrained Hardware](https://mlsanity.com/paper/2105.05983/) (2021)
- [Big-Little Adaptive Neural Networks on Low-Power Near-Subthreshold Processors](https://mlsanity.com/paper/2304.09695/) (2023)
- [Evaluating Spiking Neural Network On Neuromorphic Platform For Human Activity Recognition](https://mlsanity.com/paper/2308.00787/) (2023)
- [EdgeMark: An Automation and Benchmarking System for Embedded Artificial Intelligence Tools](https://mlsanity.com/paper/2502.01700/) (2025)
- [QPART: Adaptive Model Quantization and Dynamic Workload Balancing for Accuracy-aware Edge Inference](https://mlsanity.com/paper/2506.23934/) (2025)
- [PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation](https://mlsanity.com/paper/2603.11917/) (2026)
- [Mixed-Precision Quantization and Parallel Implementation of Multispectral Riemannian Classification for Brain--Machine Interfaces](https://mlsanity.com/paper/2102.11221/) (2021)

## Abstract (arXiv)

The growing number of low-power smart devices in the Internet of Things is coupled with the concept of "Edge Computing", that is moving some of the intelligence, especially machine learning, towards the edge of the network. Enabling machine learning algorithms to run on resource-constrained hardware, typically on low-power smart devices, is challenging in terms of hardware (optimized and energy-efficient integrated circuits), algorithmic and firmware implementations. This paper presents FANN-on-MCU, an open-source toolkit built upon the Fast Artificial Neural Network (FANN) library to run lightweight and energy-efficient neural networks on microcontrollers based on both the ARM Cortex-M series and the novel RISC-V-based Parallel Ultra-Low-Power (PULP) platform. The toolkit takes multi-layer perceptrons trained with FANN and generates code targeted at execution on low-power microcontrollers either with a floating-point unit (i.e., ARM Cortex-M4F and M7F) or without (i.e., ARM Cortex M0-M3 or PULP-based processors). This paper also provides an architectural performance evaluation of neural networks on the most popular ARM Cortex-M family and the parallel RISC-V processor called Mr. Wolf. The evaluation includes experimental results for three different applications using a self-sustainable wearable multi-sensor bracelet. Experimental results show a measured latency in the order of only a few microseconds and a power consumption of few milliwatts while keeping the memory requirements below the limitations of the targeted microcontrollers. In particular, the parallel implementation on the octa-core RISC-V platform reaches a speedup of 22x and a 69% reduction in energy consumption with respect to a single-core implementation on Cortex-M4 for continuous real-time classification.