# Enabling On-Device Smartphone GPU based Training: Lessons Learned

arXiv: https://arxiv.org/abs/2202.10100 · HTML: https://mlsanity.com/paper/2202.10100/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 23 references that are in the map.
- First posted on arXiv 21 Feb 2022.

## Builds on (most cited first, 13 of 13)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing](https://mlsanity.com/paper/1905.10083/) (2019)
- [A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1804.03294/) (2018)
- [MNN: A Universal and Efficient Inference Engine](https://mlsanity.com/paper/2002.12418/) (2020)
- [Low-Memory Neural Network Training: A Technical Report](https://mlsanity.com/paper/1904.10631/) (2019)
- [Permute, Quantize, and Fine-tune: Efficient Compression of Neural Networks](https://mlsanity.com/paper/2010.15703/) (2020)
- [Exploring System Performance of Continual Learning for Mobile and Embedded Sensing Applications](https://mlsanity.com/paper/2110.13290/) (2021)
- [FastICARL: Fast Incremental Classifier and Representation Learning with Efficient Budget Allocation in Audio Sensing Applications](https://mlsanity.com/paper/2106.07268/) (2021)
- [Knowing when we do not know: Bayesian continual learning for sensing-based analysis tasks](https://mlsanity.com/paper/2106.05872/) (2021)

## Built on by (most cited first, 4 of 4)

- [Green Edge AI: A Contemporary Survey](https://mlsanity.com/paper/2312.00333/) (2023)
- [A Two-Stage Data Selection Framework for Data-Efficient Model Training on Edge Devices](https://mlsanity.com/paper/2505.16563/) (2025)
- [Support Vector Based Anomaly Detection in Federated Learning](https://mlsanity.com/paper/2407.03920/) (2024)
- [Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator](https://mlsanity.com/paper/2607.18101/) (2026)

## Abstract (arXiv)

Deep Learning (DL) has shown impressive performance in many mobile applications. Most existing works have focused on reducing the computational and resource overheads of running Deep Neural Networks (DNN) inference on resource-constrained mobile devices. However, the other aspect of DNN operations, i.e. training (forward and backward passes) on smartphone GPUs, has received little attention thus far. To this end, we conduct an initial analysis to examine the feasibility of on-device training on smartphones using mobile GPUs. We first employ the open-source mobile DL framework (MNN) and its OpenCL backend for running compute kernels on GPUs. Next, we observed that training on CPUs is much faster than on GPUs and identified two possible bottlenecks related to this observation: (i) computation and (ii) memory bottlenecks. To solve the computation bottleneck, we optimize the OpenCL backend's kernels, showing 2x improvements (40-70 GFLOPs) over CPUs (15-30 GFLOPs) on the Snapdragon 8 series processors. However, we find that the full DNN training is still much slower on GPUs than on CPUs, indicating that memory bottleneck plays a significant role in the lower performance of GPU over CPU. The data movement takes almost 91% of training time due to the low bandwidth. Lastly, based on the findings and failures during our investigation, we present limitations and practical guidelines for future directions.