mlsanityData to 7 Oct 2026

Benchmarking of DL Libraries and Models on Mobile Devices

arXiv 2202.06512 ↗First posted 14 Feb 2022cs.LG cs.NIcited by 12

Not on the map. Why?

Abstract (arXiv)

Deploying deep learning (DL) on mobile devices has been a notable trend in recent years. To support fast inference of on-device DL, DL libraries play a critical role as algorithms and hardware do. Unfortunately, no prior work ever dives deep into the ecosystem of modern DL libs and provides quantitative results on their performance. In this paper, we first build a comprehensive benchmark that includes 6 representative DL libs and 15 diversified DL models. We then perform extensive experiments on 10 mobile devices, which help reveal a complete landscape of the current mobile DL libs ecosystem. For example, we find that the best-performing DL lib is severely fragmented across different models and hardware, and the gap between those DL libs can be rather huge. In fact, the impacts of DL libs can overwhelm the optimizations from algorithms or hardware, e.g., model quantization and GPU/DSP-based heterogeneous computing. Finally, atop the observations, we summarize practical implications to different roles in the DL lib ecosystem.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 37 of its 81 references that are in the map. First posted on arXiv 14 Feb 2022.

Builds on

37 of 81 references.

References · 37

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  7. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  8. 2018YOLOv3: An Incremental Improvementcited by 2,652
  9. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  10. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  11. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  12. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
25 more
  1. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  2. 2015Deep Learning with Limited Numerical Precisioncited by 368
  3. 2020Ansor: Generating High-Performance Tensor Programs for Deep Learningcited by 199
  4. 2019MLPerf Training Benchmarkcited by 114
  5. 2020SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloudcited by 109
  6. 2019AI Benchmark: All About Deep Learning on Smartphones in 2019cited by 76
  7. 2018AI Benchmark: Running Deep Neural Networks on Android Smartphonescited by 75
  8. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 74
  9. 2014DenseNet: Implementing Efficient ConvNet Descriptor Pyramidscited by 66
  10. 2017DeepCache: Principled Cache for Mobile Deep Visioncited by 57
  11. 2018A First Look at Deep Learning Apps on Smartphonescited by 47
  12. 2019Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networkscited by 45
  13. 2019Depthwise Convolution is All You Need for Learning Multiple Visual Domainscited by 36
  14. 2020CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Modelcited by 36
  15. 2020Chameleon: Adaptive Code Optimization for Expedited Deep Neural Network Compilationcited by 29
  16. 2021Smart at what cost? Characterising Mobile Deep Neural Networks in the wildcited by 25
  17. 2019EmBench: Quantifying Performance Variations of Deep Neural Networks across Modern Commodity Devicescited by 21
  18. 2016Fathom: Reference Workloads for Modern Deep Learning Methodscited by 20
  19. 2021It's always personal: Using Early Exits for Efficient On-Device CNN Personalisationcited by 18
  20. 2020Comparison and Benchmarking of AI Models and Frameworks on Mobile Devicescited by 17
  21. 2020Scaling Up Online Speech Recognition Using ConvNetscited by 16
  22. 2021From Cloud to Edge: A First Look at Public Edge Platformscited by 9
  23. 2020Efficient Bitwidth Search for Practical Mixed Precision Neural Networkcited by 6
  24. 2019Video Analytics with Zero-streaming Camerascited by 5
  25. 2019Approximate Query Service on Autonomous IoT Camerascited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 4; 2025: 4.

12 papers.

  1. 2023Mobile Foundation Model as Firmwarecited by 25
  2. 2022On-device Training: A First Overview on Existing Systemscited by 18
  3. 2023FrankenSplit: Efficient Neural Feature Compression with Shallow Variational Bottleneck Injection for Mobile Edge Computingcited by 12
  4. 2024Understanding Large Language Models in Your Pockets: Performance Study on COTS Mobile Devicescited by 10
  5. 2024Anatomizing Deep Learning Inference in Web Browserscited by 5
  6. 2025Leveraging Neural Graph Compilers in Machine Learning Research for Edge-Cloud Systemscited by 2
  7. 2023Impact of Deep Learning Libraries on Online Adaptive Lightweight Time Series Anomaly Detectioncited by 1
  8. 2024Impact of Recurrent Neural Networks and Deep Learning Frameworks on Real-time Lightweight Time Series Anomaly Detectioncited by 1
  9. 2025Stealthy Backdoor Attack to Real-world Models in Android Appscited by 1
  10. 2025Optimizing Multi-DNN Inference on Mobile Devices through Heterogeneous Processor Co-Executioncited by 1
  11. 2024Deep Learning Inference on Heterogeneous Mobile Processors: Potentials and Pitfallscited by 0
  12. 2025Parallax: Runtime Parallelization for Operator Fallbacks in Heterogeneous Edge Systemscited by 0