# AppBuddy: Learning to Accomplish Tasks in Mobile Apps via Reinforcement Learning

arXiv: https://arxiv.org/abs/2106.00133 · HTML: https://mlsanity.com/paper/2106.00133/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration](https://mlsanity.com/paper/1802.08802/) (2018)
- [Mapping Natural Language Instructions to Mobile UI Action Sequences](https://mlsanity.com/paper/2005.03776/) (2020)
- [Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements](https://mlsanity.com/paper/2010.04295/) (2020)
- [A Deep Hierarchical Approach to Lifelong Learning in Minecraft](https://mlsanity.com/paper/1604.07255/) (2016)
- [An empirical investigation of the challenges of real-world reinforcement learning](https://mlsanity.com/paper/2003.11881/) (2020)
- [Learning to Navigate the Web](https://mlsanity.com/paper/1812.09195/) (2018)
- [DOM-Q-NET: Grounded RL on Structured Language](https://mlsanity.com/paper/1902.07257/) (2019)
- [VASTA: A Vision and Language-assisted Smartphone Task Automation System](https://mlsanity.com/paper/1911.01474/) (2019)
- [Reinforcement Learning-Driven Test Generation for Android GUI Applications using Formal Specifications](https://mlsanity.com/paper/1911.05403/) (2019)

## Built on by (most cited first, 17 of 17)

- [AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents](https://mlsanity.com/paper/2405.14573/) (2024)
- [A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis](https://mlsanity.com/paper/2307.12856/) (2023)
- [Android in the Wild: A Large-Scale Dataset for Android Device Control](https://mlsanity.com/paper/2307.10088/) (2023)
- [Multimodal Web Navigation with Instruction-Finetuned Foundation Models](https://mlsanity.com/paper/2305.11854/) (2023)
- [A data-driven approach for learning to control computers](https://mlsanity.com/paper/2202.08137/) (2022)
- [MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agents](https://mlsanity.com/paper/2406.08184/) (2024)
- [Benchmarking Mobile Device Control Agents across Diverse Configurations](https://mlsanity.com/paper/2404.16660/) (2024)
- [Advancing Mobile GUI Agents: A Verifier-Driven Approach to Practical Deployment](https://mlsanity.com/paper/2503.15937/) (2025)
- [Mobile-Env: Building Qualified Evaluation Benchmarks for LLM-GUI Interaction](https://mlsanity.com/paper/2305.08144/) (2023)
- [OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use](https://mlsanity.com/paper/2508.04482/) (2025)
- [Foundations and Recent Trends in Multimodal Mobile Agents: A Survey](https://mlsanity.com/paper/2411.02006/) (2024)
- [Scalable Video-to-Dataset Generation for Cross-Platform Mobile Agents](https://mlsanity.com/paper/2505.12632/) (2025)
- [UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experience](https://mlsanity.com/paper/2603.24533/) (2026)
- [AutoEval: A Practical Framework for Autonomous Evaluation of Mobile Agents](https://mlsanity.com/paper/2503.02403/) (2025)
- [Video2GUI: Synthesizing Large-Scale Interaction Trajectories for Generalized GUI Agent Pretraining](https://mlsanity.com/paper/2605.14747/) (2026)
- [GhostUI: Unveiling Hidden Interactions in Mobile UI](https://mlsanity.com/paper/2601.19258/) (2026)
- [Do LLMs Need to See Everything? A Benchmark and Study of Failures in LLM-driven Smartphone Automation using Screentext vs. Screenshots](https://mlsanity.com/paper/2604.17817/) (2026)

## Abstract (arXiv)

Human beings, even small children, quickly become adept at figuring out how to use applications on their mobile devices. Learning to use a new app is often achieved via trial-and-error, accelerated by transfer of knowledge from past experiences with like apps. The prospect of building a smarter smartphone - one that can learn how to achieve tasks using mobile apps - is tantalizing. In this paper we explore the use of Reinforcement Learning (RL) with the goal of advancing this aspiration. We introduce an RL-based framework for learning to accomplish tasks in mobile apps. RL agents are provided with states derived from the underlying representation of on-screen elements, and rewards that are based on progress made in the task. Agents can interact with screen elements by tapping or typing. Our experimental results, over a number of mobile apps, show that RL agents can learn to accomplish multi-step tasks, as well as achieve modest generalization across different apps. More generally, we develop a platform which addresses several engineering challenges to enable an effective RL training environment. Our AppBuddy platform is compatible with OpenAI Gym and includes a suite of mobile apps and benchmark tasks that supports a diversity of RL research in the mobile app setting.