# Latency and Throughput Characterization of Convolutional Neural Networks for Mobile Computer Vision

arXiv: https://arxiv.org/abs/1803.09492 · HTML: https://mlsanity.com/paper/1803.09492/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2018.

## Built on by (most cited first, 13 of 13)

- [Smart at what cost? Characterising Mobile Deep Neural Networks in the wild](https://mlsanity.com/paper/2109.13963/) (2021)
- [Multi-user Co-inference with Batch Processing Capable Edge Server](https://mlsanity.com/paper/2206.06304/) (2022)
- [LEAF + AIO: Edge-Assisted Energy-Aware Object Detection for Mobile Augmented Reality](https://mlsanity.com/paper/2205.13770/) (2022)
- [Benchmarking Edge AI Platforms for High-Performance ML Inference](https://mlsanity.com/paper/2409.14803/) (2024)
- [DeepRT: A Soft Real Time Scheduler for Computer Vision Applications on the Edge](https://mlsanity.com/paper/2105.01803/) (2021)
- [On-chip Hyperspectral Image Segmentation with Fully Convolutional Networks for Scene Understanding in Autonomous Driving](https://mlsanity.com/paper/2411.19274/) (2024)
- [SMDP-Based Dynamic Batching for Efficient Inference on GPU-Based Platforms](https://mlsanity.com/paper/2301.12865/) (2023)
- [Programming with Neural Surrogates of Programs](https://mlsanity.com/paper/2112.06148/) (2021)
- [DISCO: Distributed Inference with Sparse Communications](https://mlsanity.com/paper/2302.11180/) (2023)
- [SMDP-Based Dynamic Batching for Improving Responsiveness and Energy Efficiency of Batch Services](https://mlsanity.com/paper/2501.02181/) (2025)
- [Positioning Fog Computing for Smart Manufacturing](https://mlsanity.com/paper/2205.10860/) (2022)
- [Adaptive Scheduling for Edge-Assisted DNN Serving](https://mlsanity.com/paper/2304.09961/) (2023)
- [Automated Evolutionary Optimization for Resource-Efficient Neural Network Training](https://mlsanity.com/paper/2510.09566/) (2025)

## Abstract (arXiv)

We study performance characteristics of convolutional neural networks (CNN) for mobile computer vision systems. CNNs have proven to be a powerful and efficient approach to implement such systems. However, the system performance depends largely on the utilization of hardware accelerators, which are able to speed up the execution of the underlying mathematical operations tremendously through massive parallelism. Our contribution is performance characterization of multiple CNN-based models for object recognition and detection with several different hardware platforms and software frameworks, using both local (on-device) and remote (network-side server) computation. The measurements are conducted using real workloads and real processing platforms. On the platform side, we concentrate especially on TensorFlow and TensorRT. Our measurements include embedded processors found on mobile devices and high-performance processors that can be used on the network side of mobile systems. We show that there exists significant latency--throughput trade-offs but the behavior is very complex. We demonstrate and discuss several factors that affect the performance and yield this complex behavior.