# SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers

arXiv: https://arxiv.org/abs/1905.12107 · HTML: https://mlsanity.com/paper/1905.12107/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2019.

## Built on by (most cited first, 30 of 52)

- [On-Device Training Under 256KB Memory](https://mlsanity.com/paper/2206.15472/) (2022)
- [MLPerf Tiny Benchmark](https://mlsanity.com/paper/2106.07597/) (2021)
- [MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning](https://mlsanity.com/paper/2110.15352/) (2021)
- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [Tiny Machine Learning: Progress and Futures](https://mlsanity.com/paper/2403.19076/) (2024)
- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [S2TA: Exploiting Structured Sparsity for Energy-Efficient Mobile CNN Acceleration](https://mlsanity.com/paper/2107.07983/) (2021)
- [Differentiable Network Pruning for Microcontrollers](https://mlsanity.com/paper/2110.08350/) (2021)
- [Machine Learning with Confidential Computing: A Systematization of Knowledge](https://mlsanity.com/paper/2208.10134/) (2022)
- [Ultra-Efficient On-Device Object Detection on AI-Integrated Smart Glasses with TinyissimoYOLO](https://mlsanity.com/paper/2311.01057/) (2023)
- [Intelligence at the Extreme Edge: A Survey on Reformable TinyML](https://mlsanity.com/paper/2204.00827/) (2022)
- [Leaky Nets: Recovering Embedded Neural Network Models and Inputs through Simple Power and Timing Side-Channels -- Attacks and Defenses](https://mlsanity.com/paper/2103.14739/) (2021)
- [Flexible and Fully Quantized Ultra-Lightweight TinyissimoYOLO for Ultra-Low-Power Edge Systems](https://mlsanity.com/paper/2307.05999/) (2023)
- [UDC: Unified DNAS for Compressible TinyML Models](https://mlsanity.com/paper/2201.05842/) (2022)
- [Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review](https://mlsanity.com/paper/2311.11883/) (2023)
- [Measuring what Really Matters: Optimizing Neural Networks for TinyML](https://mlsanity.com/paper/2104.10645/) (2021)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [Tiny Machine Learning for Concept Drift](https://mlsanity.com/paper/2107.14759/) (2021)
- [AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Accelerator](https://mlsanity.com/paper/2111.06503/) (2021)
- [Recognition of handwritten MNIST digits on low-memory 2 Kb RAM Arduino board using LogNNet reservoir neural network](https://mlsanity.com/paper/2105.02953/) (2021)
- [Doping: A technique for efficient compression of LSTM models using sparse structured additive matrices](https://mlsanity.com/paper/2102.07071/) (2021)
- [Environmental Sound Classification on the Edge: A Pipeline for Deep Acoustic Networks on Extremely Resource-Constrained Devices](https://mlsanity.com/paper/2103.03483/) (2021)
- [LENS: Layer Distribution Enabled Neural Architecture Search in Edge-Cloud Hierarchies](https://mlsanity.com/paper/2107.09309/) (2021)
- [TENT: Efficient Quantization of Neural Networks on the tiny Edge with Tapered FixEd PoiNT](https://mlsanity.com/paper/2104.02233/) (2021)
- [YONO: Modeling Multiple Heterogeneous Neural Networks on Microcontrollers](https://mlsanity.com/paper/2203.03794/) (2022)
- [Depth Pruning with Auxiliary Networks for TinyML](https://mlsanity.com/paper/2204.10546/) (2022)
- [PerfSAGE: Generalized Inference Performance Predictor for Arbitrary Deep Learning Models on Edge Devices](https://mlsanity.com/paper/2301.10999/) (2023)
- [The Case for Hierarchical Deep Learning Inference at the Network Edge](https://mlsanity.com/paper/2304.11763/) (2023)
- [DTMM: Deploying TinyML Models on Extremely Weak IoT Devices with Pruning](https://mlsanity.com/paper/2401.09068/) (2024)

## Abstract (arXiv)

The vast majority of processors in the world are actually microcontroller units (MCUs), which find widespread use performing simple control tasks in applications ranging from automobiles to medical devices and office equipment. The Internet of Things (IoT) promises to inject machine learning into many of these every-day objects via tiny, cheap MCUs. However, these resource-impoverished hardware platforms severely limit the complexity of machine learning models that can be deployed. For example, although convolutional neural networks (CNNs) achieve state-of-the-art results on many visual recognition tasks, CNN inference on MCUs is challenging due to severe finite memory limitations. To circumvent the memory challenge associated with CNNs, various alternatives have been proposed that do fit within the memory budget of an MCU, albeit at the cost of prediction accuracy. This paper challenges the idea that CNNs are not suitable for deployment on MCUs. We demonstrate that it is possible to automatically design CNNs which generalize well, while also being small enough to fit onto memory-limited MCUs. Our Sparse Architecture Search method combines neural architecture search with pruning in a single, unified approach, which learns superior models on four popular IoT datasets. The CNNs we find are more accurate and up to 4.35× smaller than previous approaches, while meeting the strict MCU working memory constraint.