# FastDeepIoT: Towards Understanding and Optimizing Neural Network Execution Time on Mobile and Embedded Devices

arXiv: https://arxiv.org/abs/1809.06970 · HTML: https://mlsanity.com/paper/1809.06970/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2018.

## Built on by (most cited first, 14 of 14)

- [ProgDTD: Progressive Learned Image Compression with Double-Tail-Drop Training](https://mlsanity.com/paper/2305.02145/) (2023)
- [Knowing when we do not know: Bayesian continual learning for sensing-based analysis tasks](https://mlsanity.com/paper/2106.05872/) (2021)
- [Edge-Cloud Collaborated Object Detection via Difficult-Case Discriminator](https://mlsanity.com/paper/2108.12858/) (2021)
- [Designing and Training of Lightweight Neural Networks on Edge Devices using Early Halting in Knowledge Distillation](https://mlsanity.com/paper/2209.15560/) (2022)
- [Anytime-Lidar: Deadline-aware 3D Object Detection](https://mlsanity.com/paper/2208.12181/) (2022)
- [FreeGaze: Resource-efficient Gaze Estimation via Frequency Domain Contrastive Learning](https://mlsanity.com/paper/2209.06692/) (2022)
- [MOFA: A Model Simplification Roadmap for Image Restoration on Mobile Devices](https://mlsanity.com/paper/2308.12494/) (2023)
- [On the Efficiency and Robustness of Vibration-based Foundation Models for IoT Sensing: A Case Study](https://mlsanity.com/paper/2404.02461/) (2024)
- [PriMask: Cascadable and Collusion-Resilient Data Masking for Mobile Cloud Inference](https://mlsanity.com/paper/2211.06716/) (2022)
- [Physical Self-Supervised Learning: IMU Sensing without Manual Labels](https://mlsanity.com/paper/2607.18361/) (2026)
- [Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey](https://mlsanity.com/paper/2208.10498/) (2022)
- [Efficient IoT Inference via Context-Awareness](https://mlsanity.com/paper/2310.19112/) (2023)
- [Memory-efficient Energy-adaptive Inference of Pre-Trained Models on Batteryless Embedded Systems](https://mlsanity.com/paper/2405.10426/) (2024)
- [CSRAP: Enhanced Canvas Attention Scheduling for Real-Time Mission Critical Perception](https://mlsanity.com/paper/2508.04976/) (2025)

## Abstract (arXiv)

Deep neural networks show great potential as solutions to many sensing application problems, but their excessive resource demand slows down execution time, pausing a serious impediment to deployment on low-end devices. To address this challenge, recent literature focused on compressing neural network size to improve performance. We show that changing neural network size does not proportionally affect performance attributes of interest, such as execution time. Rather, extreme run-time nonlinearities exist over the network configuration space. Hence, we propose a novel framework, called FastDeepIoT, that uncovers the non-linear relation between neural network structure and execution time, then exploits that understanding to find network configurations that significantly improve the trade-off between execution time and accuracy on mobile and embedded devices. FastDeepIoT makes two key contributions. First, FastDeepIoT automatically learns an accurate and highly interpretable execution time model for deep neural networks on the target device. This is done without prior knowledge of either the hardware specifications or the detailed implementation of the used deep learning library. Second, FastDeepIoT informs a compression algorithm how to minimize execution time on the profiled device without impacting accuracy. We evaluate FastDeepIoT using three different sensing-related tasks on two mobile devices: Nexus 5 and Galaxy Nexus. FastDeepIoT further reduces the neural network execution time by 48% to 78% and energy consumption by 37% to 69% compared with the state-of-the-art compression algorithms.