# Smart App Attack: Hacking Deep Learning Models in Android Apps

arXiv: https://arxiv.org/abs/2204.11075 · HTML: https://mlsanity.com/paper/2204.11075/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 61 references that are in the map.
- First posted on arXiv 23 Apr 2022.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [The Effectiveness of Data Augmentation in Image Classification using Deep Learning](https://mlsanity.com/paper/1712.04621/) (2017)
- [HopSkipJumpAttack: A Query-Efficient Decision-Based Attack](https://mlsanity.com/paper/1904.02144/) (2019)
- [Simple Black-box Adversarial Attacks](https://mlsanity.com/paper/1905.07121/) (2019)
- [Model-Reuse Attacks on Deep Learning Systems](https://mlsanity.com/paper/1812.00483/) (2018)
- [A First Look at Deep Learning Apps on Smartphones](https://mlsanity.com/paper/1812.05448/) (2018)
- [DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection](https://mlsanity.com/paper/2101.06896/) (2021)
- [Mind Your Weight(s): A Large-scale Study on Insufficient Machine Learning Model Protection in Mobile Apps](https://mlsanity.com/paper/2002.07687/) (2020)
- [Hybrid Batch Attacks: Finding Black-box Adversarial Examples with Limited Queries](https://mlsanity.com/paper/1908.07000/) (2019)
- [Procedural Noise Adversarial Examples for Black-Box Attacks on Deep Convolutional Networks](https://mlsanity.com/paper/1810.00470/) (2018)
- [Robustness of on-device Models: Adversarial Attack to Deep Learning Models on Android Apps](https://mlsanity.com/paper/2101.04401/) (2021)
- [Deep Learning Towards Mobile Applications](https://mlsanity.com/paper/1809.03559/) (2018)
- [A Target-Agnostic Attack on Deep Models: Exploiting Security Vulnerabilities of Transfer Learning](https://mlsanity.com/paper/1904.04334/) (2019)
- [Two Sides of the Same Coin: White-box and Black-box Attacks for Transfer Learning](https://mlsanity.com/paper/2008.11089/) (2020)
- [Detection Defense Against Adversarial Attacks with Saliency Map](https://mlsanity.com/paper/2009.02738/) (2020)
- [Adversarial Examples Versus Cloud-based Detectors: A Black-box Empirical Study](https://mlsanity.com/paper/1901.01223/) (2019)
- [Fast Differentiable Clipping-Aware Normalization and Rescaling](https://mlsanity.com/paper/2007.07677/) (2020)

## Built on by (most cited first, 12 of 12)

- [Energy-Latency Attacks to On-Device Neural Networks via Sponge Poisoning](https://mlsanity.com/paper/2305.03888/) (2023)
- [Investigating White-Box Attacks for On-Device Models](https://mlsanity.com/paper/2402.05493/) (2024)
- [Beyond the Model: Data Pre-processing Attack to Deep Learning Models in Android Apps](https://mlsanity.com/paper/2305.03963/) (2023)
- [A Picture is Worth 500 Labels: A Case Study of Demographic Disparities in Local Machine Learning Models for Instagram and TikTok](https://mlsanity.com/paper/2403.19717/) (2024)
- [THEMIS: Towards Practical Intellectual Property Protection for Post-Deployment On-Device Deep Learning Models](https://mlsanity.com/paper/2503.23748/) (2025)
- [An Empirical Study of AI Techniques in Mobile Applications](https://mlsanity.com/paper/2212.01635/) (2022)
- [Stealthy Backdoor Attack to Real-world Models in Android Apps](https://mlsanity.com/paper/2501.01263/) (2025)
- ["Impressively Scary:" Exploring User Perceptions and Reactions to Unraveling Machine Learning Models in Social Media Applications](https://mlsanity.com/paper/2503.03927/) (2025)
- [Towards Real Smart Apps: Investigating Human-AI Interactions in Smartphone On-Device AI Apps](https://mlsanity.com/paper/2307.00756/) (2023)
- [NeuroDeX: Unlocking Diverse Support in Decompiling Deep Neural Network Executables](https://mlsanity.com/paper/2509.06402/) (2025)
- [LoREnc: Low-Rank Encryption for Securing Foundation Models and LoRA Adapters](https://mlsanity.com/paper/2605.13163/) (2026)
- [SoK: Attack and Defense Landscape of Mobile On-device AI Systems](https://mlsanity.com/paper/2607.00362/) (2026)

## Abstract (arXiv)

On-device deep learning is rapidly gaining popularity in mobile applications. Compared to offloading deep learning from smartphones to the cloud, on-device deep learning enables offline model inference while preserving user privacy. However, such mechanisms inevitably store models on users' smartphones and may invite adversarial attacks as they are accessible to attackers. Due to the characteristic of the on-device model, most existing adversarial attacks cannot be directly applied for on-device models. In this paper, we introduce a grey-box adversarial attack framework to hack on-device models by crafting highly similar binary classification models based on identified transfer learning approaches and pre-trained models from TensorFlow Hub. We evaluate the attack effectiveness and generality in terms of four different settings including pre-trained models, datasets, transfer learning approaches and adversarial attack algorithms. The results demonstrate that the proposed attacks remain effective regardless of different settings, and significantly outperform state-of-the-art baselines. We further conduct an empirical study on real-world deep learning mobile apps collected from Google Play. Among 53 apps adopting transfer learning, we find that 71.7% of them can be successfully attacked, which includes popular ones in medicine, automation, and finance categories with critical usage scenarios. The results call for the awareness and actions of deep learning mobile app developers to secure the on-device models. The code of this work is available at https://github.com/Jinxhy/SmartAppAttack