mlsanityData to 14 Oct 2026

Explore, Select, Derive, and Recall: Augmenting LLM with Human-like Memory for Mobile Task Automation

arXiv 2312.03003 ↗First posted 4 Dec 2023cs.HC cs.AI cs.CLcited by 58

Not on the map. Why?

Cited by 41 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The advent of large language models (LLMs) has opened up new opportunities in the field of mobile task automation. Their superior language understanding and reasoning capabilities allow users to automate complex and repetitive tasks. However, due to the inherent unreliability and high operational cost of LLMs, their practical applicability is quite limited. To address these issues, this paper introduces MobileGPT, an innovative LLM-based mobile task automator equipped with a human-like app memory. MobileGPT emulates the cognitive process of humans interacting with a mobile app -- explore, select, derive, and recall. This approach allows for a more precise and efficient learning of a task's procedure by breaking it down into smaller, modular sub-tasks that can be re-used, re-arranged, and adapted for various objectives. We implement MobileGPT using online LLMs services (GPT-3.5 and GPT-4) and evaluate its performance on a dataset of 185 tasks across 18 mobile apps. The results indicate that MobileGPT can automate and learn new tasks with 82.7% accuracy, and is able to adapt them to different contexts with near perfect (98.75%) accuracy while reducing both latency and cost by 62.5% and 68.8%, respectively, compared to the GPT-4 powered baseline.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Builds on 26 of its 70 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

26 of 70 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  6. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  7. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  8. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  9. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  10. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  11. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 319
  12. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
14 more
  1. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  2. 2020Mapping Natural Language Instructions to Mobile UI Action Sequencescited by 181
  3. 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
  4. 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
  5. 2022Enabling Conversational Interaction with Mobile UI using Large Language Modelscited by 66
  6. 2022Spotlight: Mobile UI Understanding using Vision-Language Models with a Focuscited by 62
  7. 2021Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixelscited by 58
  8. 2023WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semanticscited by 47
  9. 2023DroidBot-GPT: GPT-powered UI Automation for Androidcited by 47
  10. 2021Screen2Vec: Semantic Embedding of GUI Screens and GUI Componentscited by 41
  11. 2022Humans decompose tasks by trading off utility and computational costcited by 18
  12. 2023Automatic Macro Mining from Interaction Traces at Scalecited by 12
  13. 2020Resource-rational Task Decomposition to Minimize Planning Costscited by 9
  14. 2022MUG: Interactive Multimodal Grounding on User Interfacescited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 30; 2026: 18.

58 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  2. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
  3. 2024Large Language Model-Brained GUI Agents: A Surveycited by 162
  4. 2024Android in the Zoo: Chain-of-Action-Thought for GUI Agentscited by 159
  5. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  6. 2024Large Multimodal Agents: A Surveycited by 105
  7. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  8. 2025ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptionscited by 45
  9. 2025Advancing Mobile GUI Agents: A Verifier-Driven Approach to Practical Deploymentcited by 38
  10. 2024LlamaTouch: A Faithful and Scalable Testbed for Mobile UI Task Automationcited by 36
  11. 2025GUI-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for GUI Agentcited by 35
  12. 2024AutoDroid-V2: Boosting SLM-based GUI Agents via Code Generationcited by 34
38 more
  1. 2025A Comprehensive Survey of Agents for Computer Use: Foundations, Challenges, and Future Directionscited by 34
  2. 2025LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospectscited by 32
  3. 2025Get Experience from Practice: LLM Agents with Record & Replaycited by 23
  4. 2025VeriSafe Agent: Safeguarding Mobile GUI Agent via Logic-based Action Verificationcited by 22
  5. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 19
  6. 2025AutoIOT: LLM-Driven Automated Natural Language Programming for AIoT Applicationscited by 17
  7. 2025AppVLM: A Lightweight Vision Language Model for Online App Controlcited by 15
  8. 2025SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agentscited by 15
  9. 2024E-ANT: A Large-Scale Dataset for Efficient Automatic GUI NavigaTioncited by 11
  10. 2025Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoCcited by 11
  11. 2025AgentProg: Empowering Long-Horizon GUI Agents with Program-Guided Context Managementcited by 11
  12. 2026MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agentscited by 11
  13. 2025MobileA3gent: Training Mobile GUI Agents Using Decentralized Self-Sourced Data from Diverse Userscited by 10
  14. 2025ProAgent: Harnessing On-Demand Sensory Contexts for Proactive LLM Agent Systems in the Wildcited by 10
  15. 2025MobiAgent: A Systematic Framework for Customizable Mobile Agentscited by 9
  16. 2024Large Language Model-Guided Semantic Alignment for Human Activity Recognitioncited by 6
  17. 2025CHOP: Mobile Operating Assistant with Constrained High-frequency Optimized Subtask Planningcited by 6
  18. 2025AI Agents for Web Testing: A Case Study in the Wildcited by 6
  19. 2025Accelerating Mobile Language Model via Speculative Decoding and NPU-Coordinated Executioncited by 5
  20. 2025CORE: Reducing UI Exposure in Mobile Agents via Collaboration Between Cloud and Local LLMscited by 5
  21. 2025Mobile-Agent-RAG: Driving Smart Multi-Agent Coordination with Contextual Knowledge Empowerment for Long-Horizon Mobile Automationcited by 5
  22. 2026PSPA-Bench: A Personalized Benchmark for Smartphone GUI Agentcited by 5
  23. 2026What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving Statescited by 5
  24. 2025Are We There Yet? A Measurement Study of Efficiency for LLM Applications on Mobile Devicescited by 4
  25. 2025Building a Stable Planner: An Extended Finite State Machine Based Planning Module for Mobile GUI Agentcited by 4
  26. 2025Towards Pervasive Distributed Agentic Generative AI -- A State of The Artcited by 4
  27. 2025GUI-Rise: Structured Reasoning and History Summarization for GUI Navigationcited by 4
  28. 2025DAF: An Efficient End-to-End Dynamic Activation Framework for on-Device DNN Trainingcited by 3
  29. 2025AFRAgent : An Adaptive Feature Renormalization Based High Resolution Aware GUI agentcited by 3
  30. 2026Jev-Mobile: Jev as an Executor for Mobile GUI Agentscited by 3
  31. 2025NGENT: Next-Generation AI Agents Must Integrate Multi-Domain Abilities to Achieve Artificial General Intelligencecited by 2
  32. 2026AutORAN: LLM-driven Natural Language Programming for Agile xApp Developmentcited by 1
  33. 2026From Volume to Value: Preference-Aligned Memory Construction for On-Device RAGcited by 1
  34. 2026Efficient On-Device Diffusion LLM Inference with Mobile NPUcited by 1
  35. 2024From Exploration to Revelation: Detecting Dark Patterns in Mobile Appscited by 0
  36. 2025Toward Autonomous UI Exploration: The UIExplorer Benchmarkcited by 0
  37. 2025MobileRAG: Enhancing Mobile Agent with Retrieval-Augmented Generationcited by 0
  38. 2026PowerLens: Taming LLM Agents for Safe and Personalized Mobile Power Managementcited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023AutoDroid: LLM-powered Task Automation in Android
  2. 2025MapAgent: Trajectory-Constructed Memory-Augmented Planning for Mobile Task Automation
  3. 2023Revolutionizing Mobile Interaction: Enabling a 3 Billion Parameter GPT LLM on Mobile
  4. 2025Task Memory Engine (TME): Enhancing State Awareness for Multi-Step LLM Agent Tasks
  5. 2024Rethinking ChatGPT's Success: Usability and Cognitive Behaviors Enabled by Auto-regressive LLMs' Prompting
  6. 2024Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents
  7. 2026Do LLMs Need to See Everything? A Benchmark and Study of Failures in LLM-driven Smartphone Automation using Screentext vs. Screenshots
  8. 2025FingerTip 20K: A Benchmark for Proactive and Personalized Mobile LLM Agents