# MLPerf Mobile Inference Benchmark

arXiv: https://arxiv.org/abs/2012.02328 · HTML: https://mlsanity.com/paper/2012.02328/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2020.

## Built on by (most cited first, 14 of 14)

- [Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices](https://mlsanity.com/paper/2310.18329/) (2023)
- [TpuGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphs](https://mlsanity.com/paper/2308.13490/) (2023)
- [MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems from Microwatts to Megawatts for Sustainable AI](https://mlsanity.com/paper/2410.12032/) (2024)
- [XRBench: An Extended Reality (XR) Machine Learning Benchmark Suite for the Metaverse](https://mlsanity.com/paper/2211.08675/) (2022)
- [DVFO: Learning-Based DVFS for Energy-Efficient Edge-Cloud Collaborative Inference](https://mlsanity.com/paper/2306.01811/) (2023)
- [The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment](https://mlsanity.com/paper/2302.06117/) (2023)
- [Sparse-DySta: Sparsity-Aware Dynamic and Static Scheduling for Sparse Multi-DNN Workloads](https://mlsanity.com/paper/2310.11096/) (2023)
- [Benchmarking Ultra-Low-Power μNPUs](https://mlsanity.com/paper/2503.22567/) (2025)
- [Reaching the Edge of the Edge: Image Analysis in Space](https://mlsanity.com/paper/2301.04954/) (2023)
- [Low-Latency Neural Inference on an Edge Device for Real-Time Handwriting Recognition from EEG Signals](https://mlsanity.com/paper/2510.19832/) (2025)
- [SONIQ: System-Optimized Noise-Injected Ultra-Low-Precision Quantization with Full-Precision Parity](https://mlsanity.com/paper/2311.14114/) (2023)
- [Combining Cloud and Mobile Computing for Machine Learning](https://mlsanity.com/paper/2402.04880/) (2024)
- [MLPerf Automotive](https://mlsanity.com/paper/2510.27065/) (2025)
- [Learning What to Evaluate: Correlation-Aware Decoupling for Multiobjective Bayesian Optimization](https://mlsanity.com/paper/2609.32632/) (2026)

## Abstract (arXiv)

This paper presents the first industry-standard open-source machine learning (ML) benchmark to allow perfor mance and accuracy evaluation of mobile devices with different AI chips and software stacks. The benchmark draws from the expertise of leading mobile-SoC vendors, ML-framework providers, and model producers. It comprises a suite of models that operate with standard data sets, quality metrics and run rules. We describe the design and implementation of this domain-specific ML benchmark. The current benchmark version comes as a mobile app for different computer vision and natural language processing tasks. The benchmark also supports non-smartphone devices, such as laptops and mobile PCs. Benchmark results from the first two rounds reveal the overwhelming complexity of the underlying mobile ML system stack, emphasizing the need for transparency in mobile ML performance analysis. The results also show that the strides being made all through the ML stack improve performance. Within six months, offline throughput improved by 3x, while latency reduced by as much as 12x. ML is an evolving field with changing use cases, models, data sets and quality targets. MLPerf Mobile will evolve and serve as an open-source community framework to guide research and innovation for mobile AI.