mlsanityData to 14 Oct 2026

MobA: Multifaceted Memory-Enhanced Adaptive Planning for Efficient Mobile Task Automation

arXiv 2410.13757 ↗First posted 17 Oct 2024cs.MA cs.AI cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

Existing Multimodal Large Language Model (MLLM)-based agents face significant challenges in handling complex GUI (Graphical User Interface) interactions on devices. These challenges arise from the dynamic and structured nature of GUI environments, which integrate text, images, and spatial relationships, as well as the variability in action spaces across different pages and tasks. To address these limitations, we propose MobA, a novel MLLM-based mobile assistant system. MobA introduces an adaptive planning module that incorporates a reflection mechanism for error recovery and dynamically adjusts plans to align with the real environment contexts and action module's execution capacity. Additionally, a multifaceted memory module provides comprehensive memory support to enhance adaptability and efficiency. We also present MobBench, a dataset designed for complex mobile interactions. Experimental results on MobBench and AndroidArena demonstrate MobA's ability to handle dynamic GUI environments and perform complex mobile tasks.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 56 of its 71 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

56 of 71 references.

References · 56

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  10. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  11. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  12. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
38 more
  1. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  2. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  3. 2023ChatDev: Communicative Agents for Software Developmentcited by 919
  4. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  5. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  6. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 563
  7. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  8. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  9. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  10. 2024Understanding the planning of LLM agents: A surveycited by 421
  11. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
  12. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
  13. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 189
  14. 2023You Only Look at Screens: Multimodal Chain-of-Action Agentscited by 185
  15. 2023AutoDroid: LLM-powered Task Automation in Androidcited by 184
  16. 2024UFO: A UI-Focused Agent for Windows OS Interactioncited by 148
  17. 2023GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigationcited by 137
  18. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 137
  19. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  20. 2021Screen2Words: Automatic Mobile UI Summarization with Multimodal Learningcited by 120
  21. 2024SEED-Data-Edit Technical Report: A Hybrid Dataset for Instructional Image Editingcited by 97
  22. 2021UIBert: Learning Generic Multimodal Representations for UI Understandingcited by 95
  23. 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
  24. 2024CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automationcited by 70
  25. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 66
  26. 2022Spotlight: Mobile UI Understanding using Vision-Language Models with a Focuscited by 62
  27. 2023PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documentscited by 61
  28. 2024Understanding the Weakness of Large Language Model Agents within a Complex Android Environmentcited by 60
  29. 2023Experiential Co-Learning of Software-Developing Agentscited by 57
  30. 2024Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?cited by 56
  31. 2023Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languagescited by 52
  32. 2024MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Trainingcited by 50
  33. 2020ActionBert: Leveraging User Actions for Semantic Understanding of User Interfacescited by 43
  34. 2021Screen2Vec: Semantic Embedding of GUI Screens and GUI Componentscited by 41
  35. 2023Mobile-Env: Building Qualified Evaluation Benchmarks for LLM-GUI Interactioncited by 36
  36. 2024MobileFlow: A Multimodal LLM For Mobile GUI Agentcited by 32
  37. 2024Investigate-Consolidate-Exploit: A General Strategy for Inter-Task Agent Self-Evolutioncited by 28
  38. 2019Interactive Task and Concept Learning from Natural Language Instructions and GUI Demonstrationscited by 26

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 6.

14 papers.

Sort:

By area:

  1. 2024Large Language Model-Brained GUI Agents: A Surveycited by 162
  2. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  3. 2025Towards Trustworthy GUI Agents: A Surveycited by 34
  4. 2025ProgRM: Build Better GUI Agents with Progress Rewardscited by 21
  5. 2026Geometry-Aware Rotary Position Embedding for Consistent Video World Modelcited by 12
  6. 2025MapAgent: Trajectory-Constructed Memory-Augmented Planning for Mobile Task Automationcited by 11
  7. 2025GUI-PRA: Process Reward Agent for GUI Taskscited by 9
  8. 2025EcoAgent: An Efficient Device-Cloud Collaborative Multi-Agent Framework for Mobile Automationcited by 8
  9. 2026AmbiBench: Benchmarking Mobile GUI Agents Beyond One-Shot Instructions in the Wildcited by 7
  10. 2026What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving Statescited by 5
  11. 2026IEA: Amateur-Friendly Conversational Image Editing Agent via Three Stages of Multitask Alignmentcited by 2
  12. 2025Robust, Observable, and Evolvable Agentic Systems Engineering: A Principled Framework Validated via the Fairy GUI Agentcited by 1
2 more
  1. 2026(A)I Sees What You Don't: Exploiting New Attack Surfaces in Third-Party Mobile Agentscited by 1
  2. 2026Learning Reliable GUI Agents under Imperfect Priorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MobileUse: A GUI Agent with Hierarchical Reflection for Autonomous Mobile Operation
  2. 2025CHOP: Mobile Operating Assistant with Constrained High-frequency Optimized Subtask Planning
  3. 2025Building a Stable Planner: An Extended Finite State Machine Based Planning Module for Mobile GUI Agent
  4. 2025Agent-SAMA: State-Aware Mobile Assistant
  5. 2026MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge Evolution
  6. 2025AppCopilot: Toward General, Accurate, Long-Horizon, and Efficient Mobile Agent
  7. 2024Dynamic Planning for LLM-based Graphical User Interface Automation
  8. 2026PalmClaw: A Native On-Device Agent Framework for Mobile Phones