# Quantization and Deployment of Deep Neural Networks on Microcontrollers

arXiv: https://arxiv.org/abs/2105.13331 · HTML: https://mlsanity.com/paper/2105.13331/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 59 references that are in the map.
- First posted on arXiv 27 May 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [MCUNet: Tiny Deep Learning on IoT Devices](https://mlsanity.com/paper/2007.10319/) (2020)
- [TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems](https://mlsanity.com/paper/2010.08678/) (2020)
- [Structured Pruning of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1512.08571/) (2015)
- [Hello Edge: Keyword Spotting on Microcontrollers](https://mlsanity.com/paper/1711.07128/) (2017)
- [Benchmarking TPU, GPU, and CPU Platforms for Deep Learning](https://mlsanity.com/paper/1907.10701/) (2019)
- [Deep k-Means: Jointly clustering with k-Means and learning representations](https://mlsanity.com/paper/1806.10069/) (2018)
- [Simultaneously Optimizing Weight and Quantizer of Ternary Neural Network using Truncated Gaussian Approximation](https://mlsanity.com/paper/1810.01018/) (2018)
- [Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)](https://mlsanity.com/paper/1807.06964/) (2018)
- [Mixed Precision DNNs: All you need is a good parametrization](https://mlsanity.com/paper/1905.11452/) (2019)
- [FANN-on-MCU: An Open-Source Toolkit for Energy-Efficient Neural Network Inference at the Edge of the Internet of Things](https://mlsanity.com/paper/1911.03314/) (2019)
- [Differentiable Mask for Pruning Convolutional and Recurrent Networks](https://mlsanity.com/paper/1909.04567/) (2019)
- [Iteratively Training Look-Up Tables for Network Quantization](https://mlsanity.com/paper/1911.04951/) (2019)
- [Attention Based Pruning for Shift Networks](https://mlsanity.com/paper/1905.12300/) (2019)

## Built on by (most cited first, 21 of 21)

- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review](https://mlsanity.com/paper/2311.11883/) (2023)
- [Understanding the effect of sparsity on neural networks robustness](https://mlsanity.com/paper/2206.10915/) (2022)
- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [An Empirical Study of Low Precision Quantization for TinyML](https://mlsanity.com/paper/2203.05492/) (2022)
- [Sharing Low Rank Conformer Weights for Tiny Always-On Ambient Speech Recognition Models](https://mlsanity.com/paper/2303.08343/) (2023)
- [Quantizing Small-Scale State-Space Models for Edge AI](https://mlsanity.com/paper/2506.12480/) (2025)
- [Shifting Capsule Networks from the Cloud to the Deep Edge](https://mlsanity.com/paper/2110.02911/) (2021)
- [Big-Little Adaptive Neural Networks on Low-Power Near-Subthreshold Processors](https://mlsanity.com/paper/2304.09695/) (2023)
- [Embedded event based object detection with spiking neural network](https://mlsanity.com/paper/2406.17617/) (2024)
- [MCU-MixQ: A HW/SW Co-optimized Mixed-precision Neural Network Design Framework for MCUs](https://mlsanity.com/paper/2407.18267/) (2024)
- [OpTC -- A Toolchain for Deployment of Neural Networks on AURIX TC3xx Microcontrollers](https://mlsanity.com/paper/2404.15833/) (2024)
- [Computer Vision Model Compression Techniques for Embedded Systems: A Survey](https://mlsanity.com/paper/2408.08250/) (2024)
- [acoupi: An Open-Source Python Framework for Deploying Bioacoustic AI Models on Edge Devices](https://mlsanity.com/paper/2501.17841/) (2025)
- [Efficient Split Learning LSTM Models for FPGA-based Edge IoT Devices](https://mlsanity.com/paper/2502.08692/) (2025)
- [MiLSD: A Micro Line-Segment Detector for Resource-Constrained Devices](https://mlsanity.com/paper/2607.06600/) (2026)
- [Deep Learning Classification of Photoplethysmogram Signal for Hypertension Levels](https://mlsanity.com/paper/2405.14556/) (2024)
- [On Reducing Activity with Distillation and Regularization for Energy Efficient Spiking Neural Networks](https://mlsanity.com/paper/2406.18350/) (2024)
- [Speech Command Recognition Using LogNNet Reservoir Computing for Embedded Systems](https://mlsanity.com/paper/2509.00862/) (2025)
- [Securing Radiation Detection Systems with an Efficient TinyML-Based IDS for Edge Devices](https://mlsanity.com/paper/2509.01592/) (2025)
- [16-bit Precision of Convolutional Neural Networks on Microcontroller Units for 8-bit Costs](https://mlsanity.com/paper/2610.03402/) (2026)

## Abstract (arXiv)

Embedding Artificial Intelligence onto low-power devices is a challenging task that has been partly overcome with recent advances in machine learning and hardware design. Presently, deep neural networks can be deployed on embedded targets to perform different tasks such as speech recognition,object detection or Human Activity Recognition. However, there is still room for optimization of deep neural networks onto embedded devices. These optimizations mainly address power consumption,memory and real-time constraints, but also an easier deployment at the edge. Moreover, there is still a need for a better understanding of what can be achieved for different use cases. This work focuses on quantization and deployment of deep neural networks onto low-power 32-bit microcontrollers. The quantization methods, relevant in the context of an embedded execution onto a microcontroller, are first outlined. Then, a new framework for end-to-end deep neural networks training, quantization and deployment is presented. This framework, called MicroAI, is designed as an alternative to existing inference engines (TensorFlow Lite for Microcontrollers and STM32CubeAI). Our framework can indeed be easily adjusted and/or extended for specific use cases. Execution using single precision 32-bit floating-point as well as fixed-point on 8- and 16-bit integers are supported. The proposed quantization method is evaluated with three different datasets (UCI-HAR, Spoken MNIST and GTSRB). Finally, a comparison study between MicroAI and both existing embedded inference engines is provided in terms of memory and power efficiency. On-device evaluation is done using ARM Cortex-M4F-based microcontrollers (Ambiq Apollo3 and STM32L452RE).