# Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

arXiv: https://arxiv.org/abs/2204.11786 · HTML: https://mlsanity.com/paper/2204.11786/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 272 of its 372 references that are in the map.
- First posted on arXiv 25 Apr 2022.

## Builds on (most cited first, 30 of 272)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)

## Built on by (most cited first, 30 of 32)

- [Fast On-device LLM Inference with NPUs](https://mlsanity.com/paper/2407.05858/) (2024)
- [Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models](https://mlsanity.com/paper/2503.06027/) (2025)
- [LLMCad: Fast and Scalable On-device Large Language Model Inference](https://mlsanity.com/paper/2309.04255/) (2023)
- [A Survey for Efficient Open Domain Question Answering](https://mlsanity.com/paper/2211.07886/) (2022)
- [Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecycle](https://mlsanity.com/paper/2303.01980/) (2023)
- [Artificial Intelligence of Things: A Survey](https://mlsanity.com/paper/2410.19998/) (2024)
- [LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platforms](https://mlsanity.com/paper/2311.11420/) (2023)
- [Training Machine Learning models at the Edge: A Survey](https://mlsanity.com/paper/2403.02619/) (2024)
- [Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Survey](https://mlsanity.com/paper/2405.01725/) (2024)
- [PLiNIO: A User-Friendly Library of Gradient-based Methods for Complexity-aware DNN Optimization](https://mlsanity.com/paper/2307.09488/) (2023)
- [Peer-to-Peer Learning Dynamics of Wide Neural Networks](https://mlsanity.com/paper/2409.15267/) (2024)
- [Enhancing Neural Architecture Search with Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devices](https://mlsanity.com/paper/2310.07217/) (2023)
- [Joint Pruning and Channel-wise Mixed-Precision Quantization for Efficient Deep Neural Networks](https://mlsanity.com/paper/2407.01054/) (2024)
- [Efficient Deep Learning Infrastructures for Embedded Computing Systems: A Comprehensive Survey and Future Envision](https://mlsanity.com/paper/2411.01431/) (2024)
- [Peer-to-Peer Deep Learning for Beyond-5G IoT](https://mlsanity.com/paper/2310.18861/) (2023)
- [DeepPicarMicro: Applying TinyML to Autonomous Cyber Physical Systems](https://mlsanity.com/paper/2208.11212/) (2022)
- [On-Device Domain Generalization](https://mlsanity.com/paper/2209.07521/) (2022)
- [Peer-to-Peer Learning + Consensus with Non-IID Data](https://mlsanity.com/paper/2312.13602/) (2023)
- [Momentum Approximation in Asynchronous Private Federated Learning](https://mlsanity.com/paper/2402.09247/) (2024)
- [Generative Artificial Intelligence for Internet of Things Computing: A Systematic Survey](https://mlsanity.com/paper/2504.07635/) (2025)
- [Energy-Aware Deep Learning on Resource-Constrained Hardware](https://mlsanity.com/paper/2505.12523/) (2025)
- [Memory-aware Scheduling for Complex Wired Networks with Iterative Graph Optimization](https://mlsanity.com/paper/2308.13898/) (2023)
- [Exploring Gen-AI applications in building research and industry: A review](https://mlsanity.com/paper/2410.01098/) (2024)
- [Optimizing 3D Gaussian Splattering for Mobile GPUs](https://mlsanity.com/paper/2511.16298/) (2025)
- [A Universal Railway Obstacle Detection System based on Semi-supervised Segmentation And Optical Flow](https://mlsanity.com/paper/2406.18908/) (2024)
- [High-Level Parallelism and Nested Features for Dynamic Inference Cost and Top-Down Attention](https://mlsanity.com/paper/2308.05128/) (2023)
- [Neural network scoring for efficient computing](https://mlsanity.com/paper/2310.09554/) (2023)
- [Machine learning's own Industrial Revolution](https://mlsanity.com/paper/2311.02278/) (2023)
- [Dynamic DNNs and Runtime Management for Efficient Inference on Mobile/Embedded Devices](https://mlsanity.com/paper/2401.08965/) (2024)
- [SPA: Towards A Computational Friendly Cloud-Base and On-Devices Collaboration Seq2seq Personalized Generation with Casual Inference](https://mlsanity.com/paper/2403.07088/) (2024)

## Abstract (arXiv)

Deep neural networks (DNNs) have achieved unprecedented success in the field of artificial intelligence (AI), including computer vision, natural language processing and speech recognition. However, their superior performance comes at the considerable cost of computational complexity, which greatly hinders their applications in many resource-constrained devices, such as mobile phones and Internet of Things (IoT) devices. Therefore, methods and techniques that are able to lift the efficiency bottleneck while preserving the high accuracy of DNNs are in great demand in order to enable numerous edge AI applications. This paper provides an overview of efficient deep learning methods, systems and applications. We start from introducing popular model compression methods, including pruning, factorization, quantization as well as compact model design. To reduce the large design cost of these manual solutions, we discuss the AutoML framework for each of them, such as neural architecture search (NAS) and automated pruning and quantization. We then cover efficient on-device training to enable user customization based on the local data on mobile devices. Apart from general acceleration techniques, we also showcase several task-specific accelerations for point cloud, video and natural language processing by exploiting their spatial sparsity and temporal/token redundancy. Finally, to support all these algorithmic advancements, we introduce the efficient deep learning system design from both software and hardware perspectives.