# Deep Learning Inference on Heterogeneous Mobile Processors: Potentials and Pitfalls

arXiv: https://arxiv.org/abs/2405.01851 · HTML: https://mlsanity.com/paper/2405.01851/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 3 May 2024.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Learning to Estimate 3D Hand Pose from Single RGB Images](https://mlsanity.com/paper/1705.01389/) (2017)
- [DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion](https://mlsanity.com/paper/2108.13342/) (2021)
- [Comparison and Benchmarking of AI Models and Frameworks on Mobile Devices](https://mlsanity.com/paper/2005.05085/) (2020)
- [Benchmarking of DL Libraries and Models on Mobile Devices](https://mlsanity.com/paper/2202.06512/) (2022)
- [DeepMix: Mobility-aware, Lightweight, and Hybrid 3D Object Detection for Headsets](https://mlsanity.com/paper/2201.08812/) (2022)
- [Sparse-DySta: Sparsity-Aware Dynamic and Static Scheduling for Sparse Multi-DNN Workloads](https://mlsanity.com/paper/2310.11096/) (2023)
- [Fast Universal Style Transfer for Artistic and Photorealistic Rendering](https://mlsanity.com/paper/1907.03118/) (2019)
- [SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference](https://mlsanity.com/paper/2303.08308/) (2023)

## Abstract (arXiv)

There is a growing demand to deploy computation-intensive deep learning (DL) models on resource-constrained mobile devices for real-time intelligent applications. Equipped with a variety of processing units such as CPUs, GPUs, and NPUs, the mobile devices hold potential to accelerate DL inference via parallel execution across heterogeneous processors. Various efficient parallel methods have been explored to optimize computation distribution, achieve load balance, and minimize communication cost across processors. Yet their practical effectiveness in the dynamic and diverse real-world mobile environment is less explored. This paper presents a holistic empirical study to assess the capabilities and challenges associated with parallel DL inference on heterogeneous mobile processors. Through carefully designed experiments covering various DL models, mobile software/hardware environments, workload patterns, and resource availability, we identify limitations of existing techniques and highlight opportunities for cross-level optimization.