# Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollers

arXiv: https://arxiv.org/abs/1905.13082 · HTML: https://mlsanity.com/paper/1905.13082/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2019.

## Built on by (most cited first, 30 of 33)

- [The case for 4-bit precision: k-bit Inference Scaling Laws](https://mlsanity.com/paper/2212.09720/) (2022)
- [On-Device Training Under 256KB Memory](https://mlsanity.com/paper/2206.15472/) (2022)
- [MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning](https://mlsanity.com/paper/2110.15352/) (2021)
- [PB-LLM: Partially Binarized Large Language Models](https://mlsanity.com/paper/2310.00034/) (2023)
- [Tiny Machine Learning: Progress and Futures](https://mlsanity.com/paper/2403.19076/) (2024)
- [Auto-Split: A General Framework of Collaborative Edge-Cloud AI](https://mlsanity.com/paper/2108.13041/) (2021)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [BiBench: Benchmarking and Analyzing Network Binarization](https://mlsanity.com/paper/2301.11233/) (2023)
- [BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer](https://mlsanity.com/paper/2105.08952/) (2021)
- [Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design](https://mlsanity.com/paper/2103.15750/) (2021)
- [Flexible and Fully Quantized Ultra-Lightweight TinyissimoYOLO for Ultra-Low-Power Edge Systems](https://mlsanity.com/paper/2307.05999/) (2023)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [Reaching the Edge of the Edge: Image Analysis in Space](https://mlsanity.com/paper/2301.04954/) (2023)
- [xTern: Energy-Efficient Ternary Neural Network Inference on RISC-V-Based Edge Systems](https://mlsanity.com/paper/2405.19065/) (2024)
- [A 3 TOPS/W RISC-V Parallel Cluster for Inference of Fine-Grain Mixed-Precision Quantized Neural Networks](https://mlsanity.com/paper/2307.01056/) (2023)
- [Free Bits: Latency Optimization of Mixed-Precision Quantized Neural Networks on the Edge](https://mlsanity.com/paper/2307.02894/) (2023)
- [MCUBERT: Memory-Efficient BERT Inference on Commodity Microcontrollers](https://mlsanity.com/paper/2410.17957/) (2024)
- [Test-Time Adaptation with Binary Feedback](https://mlsanity.com/paper/2505.18514/) (2025)
- [Rounding-Guided Backdoor Injection in Deep Learning Model Quantization](https://mlsanity.com/paper/2510.09647/) (2025)
- [FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation](https://mlsanity.com/paper/2203.12091/) (2022)
- [TinyFormer: Efficient Transformer Design and Deployment on Tiny Devices](https://mlsanity.com/paper/2311.01759/) (2023)
- [DEX: Data Channel Extension for Efficient CNN Inference on Tiny AI Accelerators](https://mlsanity.com/paper/2412.06566/) (2024)
- [EdgeMark: An Automation and Benchmarking System for Embedded Artificial Intelligence Tools](https://mlsanity.com/paper/2502.01700/) (2025)
- [Synergy: Towards On-Body AI via Tiny AI Accelerator Collaboration on Wearables](https://mlsanity.com/paper/2401.08637/) (2023)
- [TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling](https://mlsanity.com/paper/2605.09281/) (2026)
- [OpenGlass: Ultra-Low-Power On-Device AI Eyewear with Event-based Vision](https://mlsanity.com/paper/2606.07431/) (2026)
- [MiLSD: A Micro Line-Segment Detector for Resource-Constrained Devices](https://mlsanity.com/paper/2607.06600/) (2026)
- [Cost-Driven Hardware-Software Co-Optimization of Machine Learning Pipelines](https://mlsanity.com/paper/2310.07940/) (2023)
- [Value-Driven Mixed-Precision Quantization for Patch-Based Inference on Microcontrollers](https://mlsanity.com/paper/2401.13714/) (2024)
- [Tiled Bit Networks: Sub-Bit Neural Network Compression Through Reuse of Learnable Binary Vectors](https://mlsanity.com/paper/2407.12075/) (2024)

## Abstract (arXiv)

This paper presents a novel end-to-end methodology for enabling the deployment of low-error deep networks on microcontrollers. To fit the memory and computational limitations of resource-constrained edge-devices, we exploit mixed low-bitwidth compression, featuring 8, 4 or 2-bit uniform quantization, and we model the inference graph with integer-only operations. Our approach aims at determining the minimum bit precision of every activation and weight tensor given the memory constraints of a device. This is achieved through a rule-based iterative procedure, which cuts the number of bits of the most memory-demanding layers, aiming at meeting the memory constraints. After a quantization-aware retraining step, the fake-quantized graph is converted into an inference integer-only model by inserting the Integer Channel-Normalization (ICN) layers, which introduce a negligible loss as demonstrated on INT4 MobilenetV1 models. We report the latency-accuracy evaluation of mixed-precision MobilenetV1 family networks on a STM32H7 microcontroller. Our experimental results demonstrate an end-to-end deployment of an integer-only Mobilenet network with Top1 accuracy of 68% on a device with only 2MB of FLASH memory and 512kB of RAM, improving by 8% the Top1 accuracy with respect to previously published 8 bit implementations for microcontrollers.