# Smart at what cost? Characterising Mobile Deep Neural Networks in the wild

arXiv: https://arxiv.org/abs/2109.13963 · HTML: https://mlsanity.com/paper/2109.13963/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 68 references that are in the map.
- First posted on arXiv 28 Sep 2021.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [SNIP: Single-shot Network Pruning based on Connection Sensitivity](https://mlsanity.com/paper/1810.02340/) (2018)
- [ICNet for Real-Time Semantic Segmentation on High-Resolution Images](https://mlsanity.com/paper/1704.08545/) (2017)
- [Quantized Convolutional Neural Networks for Mobile Devices](https://mlsanity.com/paper/1512.06473/) (2015)
- [FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout](https://mlsanity.com/paper/2102.13451/) (2021)
- [SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud](https://mlsanity.com/paper/2008.06402/) (2020)
- [Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications](https://mlsanity.com/paper/2102.08503/) (2021)
- [AI Benchmark: All About Deep Learning on Smartphones in 2019](https://mlsanity.com/paper/1910.06663/) (2019)
- [Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions](https://mlsanity.com/paper/2106.05022/) (2021)
- [BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs](https://mlsanity.com/paper/1907.05047/) (2019)
- [FasterSeg: Searching for Faster Real-time Semantic Segmentation](https://mlsanity.com/paper/1912.10917/) (2019)
- [A First Look at Deep Learning Apps on Smartphones](https://mlsanity.com/paper/1812.05448/) (2018)
- [FSSD: Feature Fusion Single Shot Multibox Detector](https://mlsanity.com/paper/1712.00960/) (2017)
- [Mind Your Weight(s): A Large-scale Study on Insufficient Machine Learning Model Protection in Mobile Apps](https://mlsanity.com/paper/2002.07687/) (2020)
- [An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms](https://mlsanity.com/paper/1909.06727/) (2019)
- [On-Device Neural Net Inference with Mobile GPUs](https://mlsanity.com/paper/1907.01989/) (2019)
- [HAPI: Hardware-Aware Progressive Inference](https://mlsanity.com/paper/2008.03997/) (2020)
- [EmBench: Quantifying Performance Variations of Deep Neural Networks across Modern Commodity Devices](https://mlsanity.com/paper/1905.07346/) (2019)
- [It's always personal: Using Early Exits for Efficient On-Device CNN Personalisation](https://mlsanity.com/paper/2102.01393/) (2021)
- [DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device](https://mlsanity.com/paper/2104.09949/) (2021)

## Built on by (most cited first, 25 of 25)

- [EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers](https://mlsanity.com/paper/2205.03436/) (2022)
- [MELTing point: Mobile Evaluation of Language Transformers](https://mlsanity.com/paper/2403.12844/) (2024)
- [NIPQ: Noise proxy-based Integrated Pseudo-Quantization](https://mlsanity.com/paper/2206.00820/) (2022)
- [Mobile Foundation Model as Firmware](https://mlsanity.com/paper/2308.14363/) (2023)
- [PocketLLM: Enabling On-Device Fine-Tuning for Personalized LLMs](https://mlsanity.com/paper/2407.01031/) (2024)
- [Benchmarking of DL Libraries and Models on Mobile Devices](https://mlsanity.com/paper/2202.06512/) (2022)
- [The Future of Consumer Edge-AI Computing](https://mlsanity.com/paper/2210.10514/) (2022)
- [Green Federated Learning](https://mlsanity.com/paper/2303.14604/) (2023)
- [Maestro: Uncovering Low-Rank Structures via Trainable Decomposition](https://mlsanity.com/paper/2308.14929/) (2023)
- [Boosting DNN Cold Inference on Edge Devices](https://mlsanity.com/paper/2206.07446/) (2022)
- [Sparse-DySta: Sparsity-Aware Dynamic and Static Scheduling for Sparse Multi-DNN Workloads](https://mlsanity.com/paper/2310.11096/) (2023)
- [FedorAS: Federated Architecture Search under system heterogeneity](https://mlsanity.com/paper/2206.11239/) (2022)
- [Exploring the Performance and Efficiency of Transformer Models for NLP on Mobile Devices](https://mlsanity.com/paper/2306.11426/) (2023)
- [An exploration of the effect of quantisation on energy consumption and inference time of StarCoder2](https://mlsanity.com/paper/2411.12758/) (2024)
- [Edge Security: Challenges and Issues](https://mlsanity.com/paper/2206.07164/) (2022)
- [MultiTASC: A Multi-Tenancy-Aware Scheduler for Cascaded DNN Inference at the Consumer Edge](https://mlsanity.com/paper/2306.12830/) (2023)
- [A Picture is Worth 500 Labels: A Case Study of Demographic Disparities in Local Machine Learning Models for Instagram and TikTok](https://mlsanity.com/paper/2403.19717/) (2024)
- [NAWQ-SR: A Hybrid-Precision NPU Engine for Efficient On-Device Super-Resolution](https://mlsanity.com/paper/2212.09501/) (2022)
- [CARIn: Constraint-Aware and Responsive Inference on Heterogeneous Devices for Single- and Multi-DNN Workloads](https://mlsanity.com/paper/2409.01089/) (2024)
- [CoFormer: Collaborating with Heterogeneous Edge Devices for Scalable Transformer Inference](https://mlsanity.com/paper/2508.20375/) (2025)
- [Optimizing Storage Overhead of User Behavior Log for ML-embedded Mobile Apps](https://mlsanity.com/paper/2510.13405/) (2025)
- [EdgeFlowerTune: Evaluating Federated LLM Fine-Tuning Under Realistic Edge System Constraints](https://mlsanity.com/paper/2605.08636/) (2026)
- [Dynamic DNNs and Runtime Management for Efficient Inference on Mobile/Embedded Devices](https://mlsanity.com/paper/2401.08965/) (2024)
- [MultiTASC++: A Continuously Adaptive Scheduler for Edge-Based Multi-Device Cascade Inference](https://mlsanity.com/paper/2412.04147/) (2024)
- [Optimizing Feature Extraction for On-device Model Inference with User Behavior Sequences](https://mlsanity.com/paper/2603.21508/) (2026)

## Abstract (arXiv)

With smartphones' omnipresence in people's pockets, Machine Learning (ML) on mobile is gaining traction as devices become more powerful. With applications ranging from visual filters to voice assistants, intelligence on mobile comes in many forms and facets. However, Deep Neural Network (DNN) inference remains a compute intensive workload, with devices struggling to support intelligence at the cost of responsiveness.On the one hand, there is significant research on reducing model runtime requirements and supporting deployment on embedded devices. On the other hand, the strive to maximise the accuracy of a task is supported by deeper and wider neural networks, making mobile deployment of state-of-the-art DNNs a moving target. In this paper, we perform the first holistic study of DNN usage in the wild in an attempt to track deployed models and match how these run on widely deployed devices. To this end, we analyse over 16k of the most popular apps in the Google Play Store to characterise their DNN usage and performance across devices of different capabilities, both across tiers and generations. Simultaneously, we measure the models' energy footprint, as a core cost dimension of any mobile deployment. To streamline the process, we have developed gaugeNN, a tool that automates the deployment, measurement and analysis of DNNs on devices, with support for different frameworks and platforms. Results from our experience study paint the landscape of deep learning deployments on smartphones and indicate their popularity across app developers. Furthermore, our study shows the gap between bespoke techniques and real-world deployments and the need for optimised deployment of deep learning models in a highly dynamic and heterogeneous ecosystem.