# An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications

arXiv: https://arxiv.org/abs/2101.04930 · HTML: https://mlsanity.com/paper/2101.04930/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 53 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 17 of 17)

- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019)
- [DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems](https://mlsanity.com/paper/1803.07519/) (2018)
- [Taxonomy of Real Faults in Deep Learning Systems](https://mlsanity.com/paper/1910.11015/) (2019)
- [A Comprehensive Study on Deep Learning Bug Characteristics](https://mlsanity.com/paper/1906.01388/) (2019)
- [DeepCache: Principled Cache for Mobile Deep Vision](https://mlsanity.com/paper/1712.01670/) (2017)
- [Pelee: A Real-Time Object Detection System on Mobile Devices](https://mlsanity.com/paper/1804.06882/) (2018)
- [DeepBillboard: Systematic Physical-World Testing of Autonomous Driving Systems](https://mlsanity.com/paper/1812.10812/) (2018)
- [A First Look at Deep Learning Apps on Smartphones](https://mlsanity.com/paper/1812.05448/) (2018)
- [Unpaired Image-to-Image Translation using Adversarial Consistency Loss](https://mlsanity.com/paper/2003.04858/) (2020)
- [An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms](https://mlsanity.com/paper/1909.06727/) (2019)
- [Repairing Deep Neural Networks: Fix Patterns and Challenges](https://mlsanity.com/paper/2005.00972/) (2020)
- [Deep Learning Towards Mobile Applications](https://mlsanity.com/paper/1809.03559/) (2018)
- [Moving Deep Learning into Web Browser: How Far Can We Go?](https://mlsanity.com/paper/1901.09388/) (2019)
- [Emoji-Powered Representation Learning for Cross-Lingual Sentiment Classification](https://mlsanity.com/paper/1806.02557/) (2018)
- [Deep Learning for Reliable Mobile Edge Analytics in Intelligent Transportation Systems](https://mlsanity.com/paper/1712.04135/) (2017)
- [SEntiMoji: An Emoji-Powered Learning Approach for Sentiment Analysis in Software Engineering](https://mlsanity.com/paper/1907.02202/) (2019)

## Built on by (most cited first, 20 of 20)

- [Are LLMs Correctly Integrated into Software Systems?](https://mlsanity.com/paper/2407.05138/) (2024)
- [Understanding Performance Problems in Deep Learning Systems](https://mlsanity.com/paper/2112.01771/) (2021)
- [Can Agents Fix Agent Issues?](https://mlsanity.com/paper/2505.20749/) (2025)
- [Requirements Engineering for Machine Learning: A Review and Reflection](https://mlsanity.com/paper/2210.00859/) (2022)
- [Bias Behind the Wheel: Fairness Testing of Autonomous Driving Systems](https://mlsanity.com/paper/2308.02935/) (2023)
- [Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2411.19463/) (2024)
- [Anatomizing Deep Learning Inference in Web Browsers](https://mlsanity.com/paper/2402.05981/) (2024)
- [Maintainability Challenges in ML: A Systematic Literature Review](https://mlsanity.com/paper/2408.09196/) (2024)
- [Test & Evaluation Best Practices for Machine Learning-Enabled Systems](https://mlsanity.com/paper/2310.06800/) (2023)
- [Challenges in Migrating Imperative Deep Learning Programs to Graph Execution: An Empirical Study](https://mlsanity.com/paper/2201.09953/) (2022)
- [Characterizing and Understanding the Behavior of Quantized Models for Reliable Deployment](https://mlsanity.com/paper/2204.04220/) (2022)
- [Fault Localization for Buggy Deep Learning Framework Conversions in Image Recognition](https://mlsanity.com/paper/2306.06157/) (2023)
- [Deploying Deep Reinforcement Learning Systems: A Taxonomy of Challenges](https://mlsanity.com/paper/2308.12438/) (2023)
- [An Empirical Study of AI Techniques in Mobile Applications](https://mlsanity.com/paper/2212.01635/) (2022)
- [Toward Debugging Deep Reinforcement Learning Programs with RLExplorer](https://mlsanity.com/paper/2410.04322/) (2024)
- [Why Attention Fails: A Taxonomy of Faults in Attention-Based Neural Networks](https://mlsanity.com/paper/2508.04925/) (2025)
- [On Security Weaknesses and Vulnerabilities in Deep Learning Systems](https://mlsanity.com/paper/2406.08688/) (2024)
- [Speculative Automated Refactoring of Imperative Deep Learning Programs to Graph Execution](https://mlsanity.com/paper/2504.05424/) (2025)
- [Scalability and Maintainability Challenges and Solutions in Machine Learning: Systematic Literature Review](https://mlsanity.com/paper/2504.11079/) (2025)
- [AutoEmpirical: LLM-Based Automated Research for Empirical Software Fault Analysis](https://mlsanity.com/paper/2510.04997/) (2025)

## Abstract (arXiv)

Deep Learning (DL) is finding its way into a growing number of mobile software applications. These software applications, named as DL based mobile applications (abbreviated as mobile DL apps) integrate DL models trained using large-scale data with DL programs. A DL program encodes the structure of a desirable DL model and the process by which the model is trained using training data. Due to the increasing dependency of current mobile apps on DL, software engineering (SE) for mobile DL apps has become important. However, existing efforts in SE research community mainly focus on the development of DL models and extensively analyze faults in DL programs. In contrast, faults related to the deployment of DL models on mobile devices (named as deployment faults of mobile DL apps) have not been well studied. Since mobile DL apps have been used by billions of end users daily for various purposes including for safety-critical scenarios, characterizing their deployment faults is of enormous importance. To fill the knowledge gap, this paper presents the first comprehensive study on the deployment faults of mobile DL apps. We identify 304 real deployment faults from Stack Overflow and GitHub, two commonly used data sources for studying software faults. Based on the identified faults, we construct a fine-granularity taxonomy consisting of 23 categories regarding to fault symptoms and distill common fix strategies for different fault types. Furthermore, we suggest actionable implications and research avenues that could further facilitate the deployment of DL models on mobile devices.