# EmBench: Quantifying Performance Variations of Deep Neural Networks across Modern Commodity Devices

arXiv: https://arxiv.org/abs/1905.07346 · HTML: https://mlsanity.com/paper/1905.07346/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2019.

## Built on by (most cited first, 21 of 21)

- [FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout](https://mlsanity.com/paper/2102.13451/) (2021)
- [Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions](https://mlsanity.com/paper/2106.05022/) (2021)
- [MELTing point: Mobile Evaluation of Language Transformers](https://mlsanity.com/paper/2403.12844/) (2024)
- [Smart at what cost? Characterising Mobile Deep Neural Networks in the wild](https://mlsanity.com/paper/2109.13963/) (2021)
- [It's always personal: Using Early Exits for Efficient On-Device CNN Personalisation](https://mlsanity.com/paper/2102.01393/) (2021)
- [DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device](https://mlsanity.com/paper/2104.09949/) (2021)
- [Multi-Exit Semantic Segmentation Networks](https://mlsanity.com/paper/2106.03527/) (2021)
- [Benchmarking of DL Libraries and Models on Mobile Devices](https://mlsanity.com/paper/2202.06512/) (2022)
- [The Future of Consumer Edge-AI Computing](https://mlsanity.com/paper/2210.10514/) (2022)
- [Deep Neural Network-based Enhancement for Image and Video Streaming Systems: A Survey and Future Directions](https://mlsanity.com/paper/2106.03727/) (2021)
- [Benchmarking Edge AI Platforms for High-Performance ML Inference](https://mlsanity.com/paper/2409.14803/) (2024)
- [OODIn: An Optimised On-Device Inference Framework for Heterogeneous Mobile Devices](https://mlsanity.com/paper/2106.04723/) (2021)
- [A Survey of Large-Scale Deep Learning Serving System Optimization: Challenges and Opportunities](https://mlsanity.com/paper/2111.14247/) (2021)
- [FedorAS: Federated Architecture Search under system heterogeneity](https://mlsanity.com/paper/2206.11239/) (2022)
- [Sometimes Painful but Certainly Promising: Feasibility and Trade-offs of Language Model Inference at the Edge](https://mlsanity.com/paper/2503.09114/) (2025)
- [How to Reach Real-Time AI on Consumer Devices? Solutions for Programmable and Custom Architectures](https://mlsanity.com/paper/2106.15021/) (2021)
- [Adaptive Edge Offloading for Image Classification Under Rate Limit](https://mlsanity.com/paper/2208.00485/) (2022)
- [NAWQ-SR: A Hybrid-Precision NPU Engine for Efficient On-Device Super-Resolution](https://mlsanity.com/paper/2212.09501/) (2022)
- [Mitigating Memory Wall Effects in CNN Engines with On-the-Fly Weights Generation](https://mlsanity.com/paper/2307.13412/) (2023)
- [CARIn: Constraint-Aware and Responsive Inference on Heterogeneous Devices for Single- and Multi-DNN Workloads](https://mlsanity.com/paper/2409.01089/) (2024)
- [Mobiprox: Supporting Dynamic Approximate Computing on Mobiles](https://mlsanity.com/paper/2303.11291/) (2023)

## Abstract (arXiv)

In recent years, advances in deep learning have resulted in unprecedented leaps in diverse tasks spanning from speech and object recognition to context awareness and health monitoring. As a result, an increasing number of AI-enabled applications are being developed targeting ubiquitous and mobile devices. While deep neural networks (DNNs) are getting bigger and more complex, they also impose a heavy computational and energy burden on the host devices, which has led to the integration of various specialized processors in commodity devices. Given the broad range of competing DNN architectures and the heterogeneity of the target hardware, there is an emerging need to understand the compatibility between DNN-platform pairs and the expected performance benefits on each platform. This work attempts to demystify this landscape by systematically evaluating a collection of state-of-the-art DNNs on a wide variety of commodity devices. In this respect, we identify potential bottlenecks in each architecture and provide important guidelines that can assist the community in the co-design of more efficient DNNs and accelerators.