# Reduced Precision Floating-Point Optimization for Deep Neural Network On-Device Learning on MicroControllers

arXiv: https://arxiv.org/abs/2305.19167 · HTML: https://mlsanity.com/paper/2305.19167/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 30 May 2023.

## Builds on (most cited first, 23 of 23)

- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [Federated Learning in Mobile Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/1909.11875/) (2019)
- [Direct Feedback Alignment Provides Learning in Deep Neural Networks](https://mlsanity.com/paper/1609.01596/) (2016)
- [A Study of BFLOAT16 for Deep Learning Training](https://mlsanity.com/paper/1905.12322/) (2019)
- [On-Device Training Under 256KB Memory](https://mlsanity.com/paper/2206.15472/) (2022)
- [CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs](https://mlsanity.com/paper/1801.06601/) (2018)
- [Latent Replay for Real-Time Continual Learning](https://mlsanity.com/paper/1912.01100/) (2019)
- [Benchmarking TinyML Systems: Challenges and Direction](https://mlsanity.com/paper/2003.04821/) (2020)
- [Machine Learning at the Network Edge: A Survey](https://mlsanity.com/paper/1908.00080/) (2019)
- [TinyOL: TinyML with Online-Learning on Microcontrollers](https://mlsanity.com/paper/2103.08295/) (2021)
- [DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs](https://mlsanity.com/paper/2008.07127/) (2020)
- [A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays](https://mlsanity.com/paper/2110.10486/) (2021)
- [TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning](https://mlsanity.com/paper/2007.11622/) (2020)
- [Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode](https://mlsanity.com/paper/2110.09101/) (2021)
- [Accelerating RNN-based Speech Enhancement on a Multi-Core MCU with Mixed FP16-INT8 Post-Training Quantization](https://mlsanity.com/paper/2210.07692/) (2022)
- [StrassenNets: Deep Learning with a Multiplication Budget](https://mlsanity.com/paper/1712.03942/) (2017)
- [PocketNN: Integer-only Training and Inference of Neural Networks via Direct Feedback Alignment and Pocket Activations in Pure C++](https://mlsanity.com/paper/2201.02863/) (2022)
- [Distributed Low Precision Training Without Mixed Precision](https://mlsanity.com/paper/1911.07384/) (2019)
- [A transprecision floating-point cluster for efficient near-sensor data analytics](https://mlsanity.com/paper/2008.12243/) (2020)

## Built on by (most cited first, 10 of 10)

- [Training Machine Learning models at the Edge: A Survey](https://mlsanity.com/paper/2403.02619/) (2024)
- [Self-Learning for Personalized Keyword Spotting on Ultra-Low-Power Audio Sensors](https://mlsanity.com/paper/2408.12481/) (2024)
- [Enabling On-device Continual Learning with Binary Neural Networks](https://mlsanity.com/paper/2401.09916/) (2024)
- [On-device Self-supervised Learning of Visual Perception Tasks aboard Hardware-limited Nano-quadrotors](https://mlsanity.com/paper/2403.04071/) (2024)
- [Boosting keyword spotting through on-device learnable user speech characteristics](https://mlsanity.com/paper/2403.07802/) (2024)
- [Training on the Fly: On-device Self-supervised Learning aboard Nano-drones within 20 mW](https://mlsanity.com/paper/2408.03168/) (2024)
- [A Tiny Supervised ODL Core with Auto Data Pruning for Human Activity Recognition](https://mlsanity.com/paper/2408.01283/) (2024)
- [Skip2-LoRA: A Lightweight On-device DNN Fine-tuning Method for Low-cost Edge Devices](https://mlsanity.com/paper/2410.21073/) (2024)
- [Multi-modal On-Device Learning for Monocular Depth Estimation on Ultra-low-power MCUs](https://mlsanity.com/paper/2512.00086/) (2025)
- [ALADIN: Accuracy-Latency-Aware Design-space Inference Analysis for Embedded AI Accelerators](https://mlsanity.com/paper/2603.08722/) (2026)

## Abstract (arXiv)

Enabling On-Device Learning (ODL) for Ultra-Low-Power Micro-Controller Units (MCUs) is a key step for post-deployment adaptation and fine-tuning of Deep Neural Network (DNN) models in future TinyML applications. This paper tackles this challenge by introducing a novel reduced precision optimization technique for ODL primitives on MCU-class devices, leveraging the State-of-Art advancements in RISC-V RV32 architectures with support for vectorized 16-bit floating-point (FP16) Single-Instruction Multiple-Data (SIMD) operations. Our approach for the Forward and Backward steps of the Back-Propagation training algorithm is composed of specialized shape transform operators and Matrix Multiplication (MM) kernels, accelerated with parallelization and loop unrolling. When evaluated on a single training step of a 2D Convolution layer, the SIMD-optimized FP16 primitives result up to 1.72× faster than the FP32 baseline on a RISC-V-based 8+1-core MCU. An average computing efficiency of 3.11 Multiply and Accumulate operations per clock cycle (MAC/clk) and 0.81 MAC/clk is measured for the end-to-end training tasks of a ResNet8 and a DS-CNN for Image Classification and Keyword Spotting, respectively -- requiring 17.1 ms and 6.4 ms on the target platform to compute a training step on a single sample. Overall, our approach results more than two orders of magnitude faster than existing ODL software frameworks for single-core MCUs and outperforms by 1.6 × previous FP32 parallel implementations on a Continual Learning setup.