# Autonomous Large Language Model Agents Enabling Intent-Driven Mobile GUI Testing

arXiv: https://arxiv.org/abs/2311.08649 · HTML: https://mlsanity.com/paper/2311.08649/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Placed in

- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)

## Builds on (most cited first, 11 of 11)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproduction](https://mlsanity.com/paper/2209.11515/) (2022)
- [DroidBot-GPT: GPT-powered UI Automation for Android](https://mlsanity.com/paper/2304.07061/) (2023)
- [Fill in the Blank: Context-aware Automated Text Input Generation for Mobile GUI Testing](https://mlsanity.com/paper/2212.04732/) (2022)
- [Chatting with GPT-3 for Zero-Shot Human-Like Mobile Automated GUI Testing](https://mlsanity.com/paper/2305.09434/) (2023)
- [Automatic Web Testing using Curiosity-Driven Reinforcement Learning](https://mlsanity.com/paper/2103.06018/) (2021)
- [Towards Autonomous Testing Agents via Conversational Large Language Models](https://mlsanity.com/paper/2306.05152/) (2023)
- [How do Developers Test Android Applications?](https://mlsanity.com/paper/1801.06268/) (2018)
- [Improving web element localization by using a large language model](https://mlsanity.com/paper/2310.02046/) (2023)
- [DinoDroid: Testing Android Apps Using Deep Q-Networks](https://mlsanity.com/paper/2210.06307/) (2022)

## Built on by (most cited first, 15 of 15)

- [Large Language Model-Based Agents for Software Engineering: A Survey](https://mlsanity.com/paper/2409.02977/) (2024)
- [Repo2Run: Automated Building Executable Environment for Code Repository at Scale](https://mlsanity.com/paper/2502.13681/) (2025)
- [VideoGUI: A Benchmark for GUI Automation from Instructional Videos](https://mlsanity.com/paper/2406.10227/) (2024)
- [KG-RAG: Enhancing GUI Agent Decision-Making via Knowledge Graph-Driven Retrieval-Augmented Generation](https://mlsanity.com/paper/2509.00366/) (2025)
- [LLM-Explorer: Towards Efficient and Affordable LLM-based Exploration for Mobile Apps](https://mlsanity.com/paper/2505.10593/) (2025)
- [CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation](https://mlsanity.com/paper/2504.13472/) (2025)
- [AI Agents for Web Testing: A Case Study in the Wild](https://mlsanity.com/paper/2509.05197/) (2025)
- [Exploring the Integration of Large Language Models in Industrial Test Maintenance Processes](https://mlsanity.com/paper/2409.06416/) (2024)
- [GUIrilla: A Scalable Framework for Automated Desktop UI Exploration](https://mlsanity.com/paper/2510.16051/) (2025)
- [PaperRepro: Automated Computational Reproducibility Assessment for Social Science Papers](https://mlsanity.com/paper/2603.00058/) (2026)
- [Towards Test Generation from Task Description for Mobile Testing with Multi-modal Reasoning](https://mlsanity.com/paper/2504.15917/) (2025)
- [Evaluating LLM-Based Goal Extraction in Requirements Engineering: Prompting Strategies and Their Limitations](https://mlsanity.com/paper/2604.22207/) (2026)
- [DragonCrawl: A Generative, Intent-Based Framework for Scalable Mobile End-to-End Testing](https://mlsanity.com/paper/2607.28750/) (2026)
- [HxAgent: Iterative Agent Planning for End-to-End Web Application Testing](https://mlsanity.com/paper/2608.15491/) (2026)
- [When Agents See Differently: Exposing UI Desynchronization Threats in Mobile Agents](https://mlsanity.com/paper/2609.16732/) (2026)

## Abstract (arXiv)

GUI testing checks if a software system behaves as expected when users interact with its graphical interface, e.g., testing specific functionality or validating relevant use case scenarios. Currently, deciding what to test at this high level is a manual task since automated GUI testing tools target lower level adequacy metrics such as structural code coverage or activity coverage. We propose DroidAgent, an autonomous GUI testing agent for Android, for semantic, intent-driven automation of GUI testing. It is based on Large Language Models and support mechanisms such as long- and short-term memory. Given an Android app, DroidAgent sets relevant task goals and subsequently tries to achieve them by interacting with the app. Our empirical evaluation of DroidAgent using 15 apps from the Themis benchmark shows that it can set up and perform realistic tasks, with a higher level of autonomy. For example, when testing a messaging app, DroidAgent created a second account and added a first account as a friend, testing a realistic use case, without human intervention. On average, DroidAgent achieved 61% activity coverage, compared to 51% for current state-of-the-art GUI testing techniques. Further, manual analysis shows that 317 out of the 374 autonomously created tasks are realistic and relevant to app functionalities, and also that DroidAgent interacts deeply with the apps and covers more features.