# DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs

arXiv: https://arxiv.org/abs/2008.07127 · HTML: https://mlsanity.com/paper/2008.07127/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Aug 2020.

## Built on by (most cited first, 30 of 58)

- [On-Device Training Under 256KB Memory](https://mlsanity.com/paper/2206.15472/) (2022)
- [Tiny Machine Learning: Progress and Futures](https://mlsanity.com/paper/2403.19076/) (2024)
- [A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays](https://mlsanity.com/paper/2110.10486/) (2021)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode](https://mlsanity.com/paper/2110.09101/) (2021)
- [Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edge](https://mlsanity.com/paper/2301.10281/) (2023)
- [ECG-TCN: Wearable Cardiac Arrhythmia Detection with a Temporal Convolutional Network](https://mlsanity.com/paper/2103.13740/) (2021)
- [Q-PPG: Energy-Efficient PPG-based Heart Rate Monitoring on Wearable Devices](https://mlsanity.com/paper/2203.14907/) (2022)
- [On-Device Domain Learning for Keyword Spotting on Low-Power Extreme Edge Embedded Systems](https://mlsanity.com/paper/2403.10549/) (2024)
- [Flexible and Fully Quantized Ultra-Lightweight TinyissimoYOLO for Ultra-Low-Power Edge Systems](https://mlsanity.com/paper/2307.05999/) (2023)
- [Deeploy: Enabling Energy-Efficient Deployment of Small Language Models On Heterogeneous Microcontrollers](https://mlsanity.com/paper/2408.04413/) (2024)
- [Reduced Precision Floating-Point Optimization for Deep Neural Network On-Device Learning on MicroControllers](https://mlsanity.com/paper/2305.19167/) (2023)
- [Channel-wise Mixed-precision Assignment for DNN Inference on Constrained Edge Nodes](https://mlsanity.com/paper/2206.08852/) (2022)
- [Distributed On-Sensor Compute System for AR/VR Devices: A Semi-Analytical Simulation Framework for Power Estimation](https://mlsanity.com/paper/2203.07474/) (2022)
- [Marsellus: A Heterogeneous RISC-V AI-IoT End-Node SoC with 2-to-8b DNN Acceleration and 30%-Boost Adaptive Body Biasing](https://mlsanity.com/paper/2305.08415/) (2023)
- [Optimizing the Deployment of Tiny Transformers on Low-Power MCUs](https://mlsanity.com/paper/2404.02945/) (2024)
- [Enhancing Neural Architecture Search with Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devices](https://mlsanity.com/paper/2310.07217/) (2023)
- [A Tiny Transformer for Low-Power Arrhythmia Classification on Microcontrollers](https://mlsanity.com/paper/2402.10748/) (2024)
- [MATCH: Model-Aware TVM-based Compilation for Heterogeneous Edge Devices](https://mlsanity.com/paper/2410.08855/) (2024)
- [TCN Mapping Optimization for Ultra-Low Power Time-Series Edge Inference](https://mlsanity.com/paper/2203.12925/) (2022)
- [Adaptive Deep Learning for Efficient Visual Pose Estimation aboard Ultra-low-power Nano-drones](https://mlsanity.com/paper/2401.15236/) (2024)
- [xTern: Energy-Efficient Ternary Neural Network Inference on RISC-V-Based Edge Systems](https://mlsanity.com/paper/2405.19065/) (2024)
- [HW-Aware Initialization of DNN Auto-Tuning to Improve Exploration Time and Robustness](https://mlsanity.com/paper/2205.15568/) (2022)
- [Deep Neural Network Architecture Search for Accurate Visual Pose Estimation aboard Nano-UAVs](https://mlsanity.com/paper/2303.01931/) (2023)
- [Precision-aware Latency and Energy Balancing on Multi-Accelerator Platforms for DNN Inference](https://mlsanity.com/paper/2306.05060/) (2023)
- [Dynamic Decision Tree Ensembles for Energy-Efficient Inference on IoT Edge Nodes](https://mlsanity.com/paper/2306.09789/) (2023)
- [A 3 TOPS/W RISC-V Parallel Cluster for Inference of Fine-Grain Mixed-Precision Quantized Neural Networks](https://mlsanity.com/paper/2307.01056/) (2023)
- [Free Bits: Latency Optimization of Mixed-Precision Quantized Neural Networks on the Edge](https://mlsanity.com/paper/2307.02894/) (2023)
- [12 mJ per Class On-Device Online Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2403.07851/) (2024)
- [A Map-free Deep Learning-based Framework for Gate-to-Gate Monocular Visual Navigation aboard Miniaturized Aerial Vehicles](https://mlsanity.com/paper/2503.05251/) (2025)

## Abstract (arXiv)

The deployment of Deep Neural Networks (DNNs) on end-nodes at the extreme edge of the Internet-of-Things is a critical enabler to support pervasive Deep Learning-enhanced applications. Low-Cost MCU-based end-nodes have limited on-chip memory and often replace caches with scratchpads, to reduce area overheads and increase energy efficiency -- requiring explicit DMA-based memory transfers between different levels of the memory hierarchy. Mapping modern DNNs on these systems requires aggressive topology-dependent tiling and double-buffering. In this work, we propose DORY (Deployment Oriented to memoRY) - an automatic tool to deploy DNNs on low cost MCUs with typically less than 1MB of on-chip SRAM memory. DORY abstracts tiling as a Constraint Programming (CP) problem: it maximizes L1 memory utilization under the topological constraints imposed by each DNN layer. Then, it generates ANSI C code to orchestrate off- and on-chip transfers and computation phases. Furthermore, to maximize speed, DORY augments the CP formulation with heuristics promoting performance-effective tile sizes. As a case study for DORY, we target GreenWaves Technologies GAP8, one of the most advanced parallel ultra-low power MCU-class devices on the market. On this device, DORY achieves up to 2.5x better MAC/cycle than the GreenWaves proprietary software solution and 18.1x better than the state-of-the-art result on an STM32-F746 MCU on single layers. Using our tool, GAP-8 can perform end-to-end inference of a 1.0-MobileNet-128 network consuming just 63 pJ/MAC on average @ 4.3 fps - 15.4x better than an STM32-F746. We release all our developments - the DORY framework, the optimized backend kernels, and the related heuristics - as open-source software.