mlsanityData to 14 Oct 2026

AutoDroid: LLM-powered Task Automation in Android

arXiv 2308.15272 ↗First posted 29 Aug 2023cs.AI cs.SEcited by 184

Not on the map. Why?

Cited by 112 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Mobile task automation is an attractive technique that aims to enable voice-based hands-free user interaction with smartphones. However, existing approaches suffer from poor scalability due to the limited language understanding ability and the non-trivial manual efforts required from developers or end-users. The recent advance of large language models (LLMs) in language understanding and reasoning inspires us to rethink the problem from a model-centric perspective, where task preparation, comprehension, and execution are handled by a unified language model. In this work, we introduce AutoDroid, a mobile task automation system capable of handling arbitrary tasks on any Android application without manual efforts. The key insight is to combine the commonsense knowledge of LLMs and domain-specific knowledge of apps through automated dynamic analysis. The main components include a functionality-aware UI representation method that bridges the UI with the LLM, exploration-based memory injection techniques that augment the app-specific domain knowledge of LLM, and a multi-granularity query optimization module that reduces the cost of model inference. We integrate AutoDroid with off-the-shelf LLMs including online GPT-4/GPT-3.5 and on-device Vicuna, and evaluate its performance on a new benchmark for memory-augmented Android task automation with 158 common tasks. The results demonstrated that AutoDroid is able to precisely generate actions with an accuracy of 90.9%, and complete tasks with a success rate of 71.3%, outperforming the GPT-4-powered baselines by 36.4% and 39.7%. The demo, benchmark suites, and source code of AutoDroid will be released at url{https://autodroid-sys.github.io/}.

Key facts. Cited by 184 papers in the map (data through 14 Oct 2026). Builds on 24 of its 45 references that are in the map. First posted on arXiv 29 Aug 2023.

Builds on

24 of 45 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  8. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  9. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  10. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  11. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
  12. 2020Mapping Natural Language Instructions to Mobile UI Action Sequencescited by 181
12 more
  1. 2021Screen2Words: Automatic Mobile UI Summarization with Multimodal Learningcited by 120
  2. 2020Widget Captioning: Generating Natural Language Description for Mobile User Interface Elementscited by 95
  3. 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
  4. 2022A data-driven approach for learning to control computerscited by 88
  5. 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
  6. 2022Enabling Conversational Interaction with Mobile UI using Large Language Modelscited by 66
  7. 2022Spotlight: Mobile UI Understanding using Vision-Language Models with a Focuscited by 62
  8. 2022A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibilitycited by 61
  9. 2021Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixelscited by 58
  10. 2023DroidBot-GPT: GPT-powered UI Automation for Androidcited by 47
  11. 2021VUT: Versatile UI Transformer for Multi-Modal Multi-Task User Interface Modelingcited by 26
  12. 2023Responsible Task Automation: Empowering Large Language Models as Responsible Task Automatorscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 35; 2025: 87; 2026: 59.

184 papers.

Sort:

By area:

  1. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  2. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  3. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
  4. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 189
  5. 2024Android in the Zoo: Chain-of-Action-Thought for GUI Agentscited by 159
  6. 2024Aria-UI: Visual Grounding for GUI Instructionscited by 140
  7. 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasonerscited by 140
  8. 2023GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigationcited by 137
  9. 2024GUI Agents with Foundation Models: A Comprehensive Surveycited by 111
  10. 2024GUI Agents: A Surveycited by 103
  11. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 86
  12. 2024Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systemscited by 81
38 more
  1. 2024SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluationcited by 81
  2. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 81
  3. 2024AppAgent v2: Advanced Agent for Flexible Mobile Interactionscited by 68
  4. 2025InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflectioncited by 67
  5. 2024Fast On-device LLM Inference with NPUscited by 62
  6. 2025A Survey on (M)LLM-Based GUI Agentscited by 62
  7. 2024Ferret-UI 2: Mastering Universal User Interface Understanding Across Platformscited by 51
  8. 2025AppAgentX: Evolving GUI Agents as Proficient Smartphone Userscited by 46
  9. 2023ASSISTGUI: Task-Oriented Desktop Graphical User Interface Automationcited by 45
  10. 2025ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptionscited by 45
  11. 2025ZeroGUI: Automating Online GUI Learning at Zero Human Costcited by 44
  12. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 42
  13. 2025Advancing Mobile GUI Agents: A Verifier-Driven Approach to Practical Deploymentcited by 38
  14. 2025ViMo: A Generative Visual GUI World Model for App Agentscited by 35
  15. 2025GUI-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for GUI Agentcited by 35
  16. 2024AutoDroid-V2: Boosting SLM-based GUI Agents via Code Generationcited by 34
  17. 2025A Comprehensive Survey of Agents for Computer Use: Foundations, Challenges, and Future Directionscited by 34
  18. 2025Towards Trustworthy GUI Agents: A Surveycited by 34
  19. 2025LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospectscited by 32
  20. 2024Foundations and Recent Trends in Multimodal Mobile Agents: A Surveycited by 31
  21. 2024MobileExperts: A Dynamic Tool-Enabled Agent Team in Mobile Devicescited by 29
  22. 2025Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error Diagnosis in GUI Automationcited by 27
  23. 2025GUI-Xplore: Empowering Generalizable GUI Agents with One Explorationcited by 25
  24. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?cited by 23
  25. 2025WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Pointcited by 20
  26. 2025FingerTip 20K: A Benchmark for Proactive and Personalized Mobile LLM Agentscited by 20
  27. 2024MobileAgent: enhancing mobile control via human-machine interaction and SOP integrationcited by 18
  28. 2025MobileWorldBench: Towards Semantic World Modeling For Mobile Agentscited by 18
  29. 2025AutoIOT: LLM-Driven Automated Natural Language Programming for AIoT Applicationscited by 17
  30. 2025MobileSteward: Integrating Multiple App-Oriented Agents with Self-Evolution to Automate Cross-App Instructionscited by 16
  31. 2025Mobile-Bench-v2: A More Realistic and Comprehensive Benchmark for VLM-based Mobile Agentscited by 16
  32. 2025AppVLM: A Lightweight Vision Language Model for Online App Controlcited by 15
  33. 2025SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agentscited by 15
  34. 2025Uncertainty-Aware GUI Agent: Adaptive Perception through Component Recommendation and Human-in-the-Loop Refinementcited by 15
  35. 2025EchoTrail-GUI: Building Actionable Memory for GUI Agents via Critic-Guided Self-Explorationcited by 15
  36. 2024MobA: Multifaceted Memory-Enhanced Adaptive Planning for Efficient Mobile Task Automationcited by 14
  37. 2024Lightweight Neural App Controlcited by 12
  38. 2025Agent-SAMA: State-Aware Mobile Assistantcited by 12

50 most cited of 184 All 184 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Explore, Select, Derive, and Recall: Augmenting LLM with Human-like Memory for Mobile Task Automation
  2. 2025LELANTE: LEveraging LLM for Automated ANdroid TEsting ↗
  3. 2024Understanding the Weakness of Large Language Model Agents within a Complex Android Environment
  4. 2025AndroidGen: Building an Android Language Agent under Data Scarcity
  5. 2026GraphPilot: GUI Task Automation with One-Step LLM Reasoning Powered by Knowledge Graph
  6. 2025A Pilot Study on LLM-Based Agentic Translation from Android to iOS: Pitfalls and Insights
  7. 2023Revolutionizing Mobile Interaction: Enabling a 3 Billion Parameter GPT LLM on Mobile
  8. 2023Mobile-Env: Building Qualified Evaluation Benchmarks for LLM-GUI Interaction