mlsanityData to 14 Oct 2026

Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications

arXiv 2508.15008 ↗First posted 20 Aug 2025cs.LG cs.AI cs.ARcited by 7

Not on the map. Why?

Abstract (arXiv)

The deployment of Quantized Neural Networks (QNNs) on resource-constrained edge devices, such as microcontrollers (MCUs), introduces fundamental challenges in balancing model performance, computational complexity, and memory constraints. Tiny Machine Learning (TinyML) addresses these issues by jointly advancing machine learning algorithms, hardware architectures, and software optimization techniques to enable deep neural network inference on embedded systems. This survey provides a hardware-oriented perspective on neural network quantization, systematically reviewing the quantization methods most relevant to MCUs and extreme-edge devices. Particular emphasis is placed on the critical trade-offs between model performance and the capabilities of MCU-class hardware, including memory hierarchies, numerical representations, and accelerator support. The survey further reviews contemporary MCU hardware platforms, including ARM-based and RISC-V-based designs, as well as MCUs integrating neural processing units (NPUs) for low-precision inference, together with the supporting software stacks. In addition, we analyze real-world deployments of quantized models on MCUs and consolidate the application domains in which such systems are used. Finally, we discuss open challenges and outline promising future directions toward scalable, energy-efficient, and sustainable AI deployment on edge devices.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 168 of its 322 references that are in the map. First posted on arXiv 20 Aug 2025.

Builds on

168 of 322 references.

References · 168

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018YOLOv3: An Incremental Improvementcited by 2,659
  4. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  5. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  6. 2020Knowledge Distillation: A Surveycited by 1,603
  7. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  8. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  9. 2017Mixed Precision Trainingcited by 1,156
  10. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  11. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  12. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
38 more
  1. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  2. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  3. 2021A White Paper on Neural Network Quantizationcited by 472
  4. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  5. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  6. 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 434
  7. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  8. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  9. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  10. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 379
  11. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  12. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 359
  13. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  14. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 274
  15. 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 273
  16. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  17. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  18. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 254
  19. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 238
  20. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
  21. 2016Trained Ternary Quantizationcited by 208
  22. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 208
  23. 2016Ternary Weight Networkscited by 204
  24. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 193
  25. 2020Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluationcited by 188
  26. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  27. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  28. 2020TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systemscited by 165
  29. 2019Quantization Networkscited by 153
  30. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 151
  31. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 151
  32. 2015Fixed Point Quantization of Deep Convolutional Networkscited by 143
  33. 2018CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUscited by 121
  34. 2017Deep Learning with Low Precision by Half-wave Gaussian Quantizationcited by 117
  35. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
  36. 2016Convolutional Neural Networks using Logarithmic Data Representationcited by 99
  37. 2020APQ: Joint Search for Network Architecture, Pruning and Quantization Policycited by 99
  38. 2021A Survey of Modern Deep Learning based Object Detection Modelscited by 98

50 most cited of 168 All 168 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2026What Survives When You Compress a Recursive Reasoner for the Edge?cited by 3
  2. 2025Link-Aware Energy-Frugal Continual Learning for Fault Detection in IoT Networkscited by 1
  3. 2026Not All Objectives Are Born Equal: Priority-Constrained Descent for Hierarchical Multi-Objective Optimizationcited by 1
  4. 2025Low-Bit, High-Fidelity: Optimal Transport Quantization for Flow Matchingcited by 0
  5. 2026TinyNeRV: Compact Neural Video Representations via Capacity Scaling, Distillation, and Low-Precision Inferencecited by 0
  6. 2026Different Strokes for Different Folks: Writer Identification for Historical Arabic Manuscriptscited by 0
  7. 202616-bit Precision of Convolutional Neural Networks on Microcontroller Units for 8-bit Costscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review
  2. 2025From Tiny Machine Learning to Tiny Deep Learning: A Survey
  3. 2021Neural Network Quantization for Efficient Inference: A Survey
  4. 2023Towards Efficient In-memory Computing Hardware for Quantized Neural Networks: State-of-the-art, Open Challenges and Perspectives
  5. 2022AMED: Automatic Mixed-Precision Quantization for Edge Devices
  6. 2025Evaluating the Energy Efficiency of NPU-Accelerated Machine Learning Inference on Embedded Microcontrollers
  7. 2024David and Goliath: An Empirical Evaluation of Attacks and Defenses for QNNs at the Deep Edge
  8. 2024Value-Driven Mixed-Precision Quantization for Patch-Based Inference on Microcontrollers