# Deep Reinforcement Learning for Black-Box Testing of Android Apps

arXiv: https://arxiv.org/abs/2101.02636 · HTML: https://mlsanity.com/paper/2101.02636/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2021.

## Built on by (most cited first, 10 of 10)

- [LLM-Explorer: Towards Efficient and Affordable LLM-based Exploration for Mobile Apps](https://mlsanity.com/paper/2505.10593/) (2025)
- [Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detection](https://mlsanity.com/paper/2506.18245/) (2025)
- [SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity](https://mlsanity.com/paper/2602.08690/) (2026)
- [A Comparison of Reinforcement Learning Frameworks for Software Testing Tasks](https://mlsanity.com/paper/2208.12136/) (2022)
- [Hawkeye: Change-targeted Testing for Android Apps based on Deep Reinforcement Learning](https://mlsanity.com/paper/2309.01519/) (2023)
- [Fidelity-Induced Interpretable Policy Extraction for Reinforcement Learning](https://mlsanity.com/paper/2309.06097/) (2023)
- [Requirement-Based Testing: Enhancing Reinforcement Learning with Game Theory](https://mlsanity.com/paper/2407.18994/) (2024)
- [CovAgent: Overcoming the 30% Curse of Mobile Application Coverage with Agentic AI and Dynamic Instrumentation](https://mlsanity.com/paper/2601.21253/) (2026)
- [Can LLMs Test Terminal User Interfaces?](https://mlsanity.com/paper/2608.03743/) (2026)
- [RotDroid: Cross-Orientation State Equivalence Testing for Detecting GUI Rotation Bugs in Android Apps](https://mlsanity.com/paper/2608.25425/) (2026)

## Abstract (arXiv)

The state space of Android apps is huge and its thorough exploration during testing remains a major challenge. In fact, the best exploration strategy is highly dependent on the features of the app under test. Reinforcement Learning (RL) is a machine learning technique that learns the optimal strategy to solve a task by trial and error, guided by positive or negative reward, rather than by explicit supervision. Deep RL is a recent extension of RL that takes advantage of the learning capabilities of neural networks. Such capabilities make Deep RL suitable for complex exploration spaces such as the one of Android apps. However, state of the art, publicly available tools only support basic, tabular RL. We have developed ARES, a Deep RL approach for black-box testing of Android apps. Experimental results show that it achieves higher coverage and fault revelation than the baselines, which include state of the art RL based tools, such as TimeMachine and Q-Testing. We also investigated qualitatively the reasons behind such performance and we have identified the key features of Android apps that make Deep RL particularly effective on them to be the presence of chained and blocking activities.