# Neural Network Inference on Mobile SoCs

arXiv: https://arxiv.org/abs/1908.11450 · HTML: https://mlsanity.com/paper/1908.11450/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2019.

## Built on by (most cited first, 17 of 17)

- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning](https://mlsanity.com/paper/2107.08147/) (2021)
- [Smart at what cost? Characterising Mobile Deep Neural Networks in the wild](https://mlsanity.com/paper/2109.13963/) (2021)
- [FLASH: Fast Neural Architecture Search with Hardware Optimization](https://mlsanity.com/paper/2108.00568/) (2021)
- [Efficient Softmax Approximation for Deep Neural Networks with Attention Mechanism](https://mlsanity.com/paper/2111.10770/) (2021)
- [Accelerating Neural Network Inference with Processing-in-DRAM: From the Edge to the Cloud](https://mlsanity.com/paper/2209.08938/) (2022)
- [Dynamic DNN Decomposition for Lossless Synergistic Inference](https://mlsanity.com/paper/2101.05952/) (2021)
- [Precision-aware Latency and Energy Balancing on Multi-Accelerator Platforms for DNN Inference](https://mlsanity.com/paper/2306.05060/) (2023)
- [The Role of Transformer Models in Advancing Blockchain Technology: A Systematic Survey](https://mlsanity.com/paper/2409.02139/) (2024)
- [Adaptive Edge Offloading for Image Classification Under Rate Limit](https://mlsanity.com/paper/2208.00485/) (2022)
- [PointSplit: Towards On-device 3D Object Detection with Heterogeneous Low-power Accelerators](https://mlsanity.com/paper/2504.03654/) (2025)
- [NAWQ-SR: A Hybrid-Precision NPU Engine for Efficient On-Device Super-Resolution](https://mlsanity.com/paper/2212.09501/) (2022)
- [NeuroFlux: Memory-Efficient CNN Training Using Adaptive Local Learning](https://mlsanity.com/paper/2402.14139/) (2024)
- [Lightweight and Flexible Deep Equilibrium Learning for CSI Feedback in FDD Massive MIMO](https://mlsanity.com/paper/2211.15079/) (2022)
- [Comparative Study of Parameter Selection for Enhanced Edge Inference for a Multi-Output Regression model for Head Pose Estimation](https://mlsanity.com/paper/2302.00592/) (2022)
- [Optimizing DNN Inference on Multi-Accelerator SoCs at Training-time](https://mlsanity.com/paper/2409.18566/) (2024)
- [Power-Performance Characterization of TinyML Systems](https://mlsanity.com/paper/2608.21646/) (2026)

## Abstract (arXiv)

The ever-increasing demand from mobile Machine Learning (ML) applications calls for evermore powerful on-chip computing resources. Mobile devices are empowered with heterogeneous multi-processor Systems-on-Chips (SoCs) to process ML workloads such as Convolutional Neural Network (CNN) inference. Mobile SoCs house several different types of ML capable components on-die, such as CPU, GPU, and accelerators. These different components are capable of independently performing inference but with very different power-performance characteristics. In this article, we provide a quantitative evaluation of the inference capabilities of the different components on mobile SoCs. We also present insights behind their respective power-performance behavior. Finally, we explore the performance limit of the mobile SoCs by synergistically engaging all the components concurrently. We observe that a mobile SoC provides up to 2x improvement with parallel inference when all its components are engaged, as opposed to engaging only one component.