mlsanityData to 14 Oct 2026

Differentiable Network Pruning for Microcontrollers

arXiv 2110.08350 ↗First posted 15 Oct 2021cs.LG cs.SY eess.SYcited by 16

Not on the map. Why?

Abstract (arXiv)

Embedded and personal IoT devices are powered by microcontroller units (MCUs), whose extreme resource scarcity is a major obstacle for applications relying on on-device deep learning inference. Orders of magnitude less storage, memory and computational capacity, compared to what is typically required to execute neural networks, impose strict structural constraints on the network architecture and call for specialist model compression methodology. In this work, we present a differentiable structured network pruning method for convolutional neural networks, which integrates a model's MCU-specific resource usage and parameter importance feedback to obtain highly compressed yet accurate classification models. Our methodology (a) improves key resource usage of models up to 80x; (b) prunes iteratively while a model is trained, resulting in little to no overhead or even improved training time; (c) produces compressed models with matching or improved resource usage up to 1.4x in less time compared to prior MCU-specific methods. Compressed models are available for download.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 38 of its 67 references that are in the map. First posted on arXiv 15 Oct 2021.

Builds on

38 of 67 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  7. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  8. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  9. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  10. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
  11. 2020What is the State of Neural Network Pruning?cited by 583
  12. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
26 more
  1. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  2. 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 273
  3. 2014Efficient Object Localization Using Convolutional Networkscited by 227
  4. 2020TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systemscited by 165
  5. 2015Structured Pruning of Deep Convolutional Neural Networkscited by 149
  6. 2020Dynamic Model Pruning with Feedbackcited by 125
  7. 2018CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUscited by 121
  8. 2020BRP-NAS: Prediction-based NAS using GCNscited by 111
  9. 2017Hello Edge: Keyword Spotting on Microcontrollerscited by 110
  10. 2018A Flexible Framework for Multi-Objective Bayesian Optimization using Random Scalarizationscited by 99
  11. 2017MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networkscited by 94
  12. 2018Faster gaze prediction with dense networks and Fisher pruningcited by 94
  13. 2017Deep Residual Learning for Small-Footprint Keyword Spottingcited by 68
  14. 2020μNAS: Constrained Neural Architecture Search for Microcontrollerscited by 55
  15. 2019SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollerscited by 52
  16. 2019AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Ratescited by 49
  17. 2020DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocationcited by 47
  18. 2019Visual Wake Words Datasetcited by 46
  19. 2018Structured Pruning of Neural Networks with Budget-Aware Regularizationcited by 26
  20. 2020Single Shot Structured Pruning Before Trainingcited by 19
  21. 2020Attend And Discriminate: Beyond the State-of-the-Art for Human Activity Recognition using Wearable Sensorscited by 19
  22. 2021DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Devicecited by 18
  23. 2019Neural networks on microcontrollers: saving memory at inference via operator reorderingcited by 15
  24. 2019SwiftNet: Using Graph Propagation as Meta-knowledge to Search Highly Representative Neural Architecturescited by 5
  25. 2019CoopNet: Cooperative Convolutional Neural Network for Low-Power MCUscited by 4
  26. 2017Cross-modal Recurrent Models for Weight Objective Prediction from Multimodal Time-series Datacited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 3; 2025: 2; 2026: 3.

16 papers.

Sort:

  1. 2022Machine Learning for Microcontroller-Class Hardware: A Reviewcited by 28
  2. 2023Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiencescited by 21
  3. 2023TinyML: Tools, Applications, Challenges, and Future Research Directionscited by 14
  4. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  5. 2023MicroNAS: Memory and Latency Constrained Hardware-Aware Neural Architecture Search for Time Series Classification on Microcontrollerscited by 8
  6. 2022Pex: Memory-efficient Microcontroller Deep Learning through Partial Executioncited by 3
  7. 2025Test-Time Adaptation with Binary Feedbackcited by 3
  8. 2024DEX: Data Channel Extension for Efficient CNN Inference on Tiny AI Acceleratorscited by 2
  9. 2025γ-Quant: Towards Learnable Quantization for Low-bit Pattern Recognitioncited by 2
  10. 2023Synergy: Towards On-Body AI via Tiny AI Accelerator Collaboration on Wearablescited by 1
  11. 2023Cost-Driven Hardware-Software Co-Optimization of Machine Learning Pipelinescited by 0
  12. 2024Value Prediction for Spatiotemporal Gait Data Using Deep Learningcited by 0
4 more
  1. 2024DiTMoS: Delving into Diverse Tiny-Model Selection on Microcontrollerscited by 0
  2. 2026Scaling Laws in the Tiny Regime: How Small Models Change Their Mistakescited by 0
  3. 2026Embedded Arena: Iterative Optimization via Hardware Feedbackcited by 0
  4. 202616-bit Precision of Convolutional Neural Networks on Microcontroller Units for 8-bit Costscited by 0