# MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments

arXiv: https://arxiv.org/abs/2602.06075 · HTML: https://mlsanity.com/paper/2602.06075/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2026.

## Built on by (most cited first, 18 of 18)

- [SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization](https://mlsanity.com/paper/2604.02268/) (2026)
- [Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents](https://mlsanity.com/paper/2604.10674/) (2026)
- [Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents](https://mlsanity.com/paper/2602.16855/) (2026)
- [KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation](https://mlsanity.com/paper/2604.08455/) (2026)
- [MobileForge: Annotation-Free Adaptation for Mobile GUI Agents with Hierarchical Feedback-Guided Policy Optimization](https://mlsanity.com/paper/2606.19930/) (2026)
- [π-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows](https://mlsanity.com/paper/2605.14678/) (2026)
- [What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving States](https://mlsanity.com/paper/2606.31612/) (2026)
- [UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization](https://mlsanity.com/paper/2604.13822/) (2026)
- [MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Management](https://mlsanity.com/paper/2606.19926/) (2026)
- [DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injection](https://mlsanity.com/paper/2606.27499/) (2026)
- [A Task-State Representation for Long-Horizon Mobile GUI Agents](https://mlsanity.com/paper/2607.00502/) (2026)
- [UI-Venus-2 Technical Report](https://mlsanity.com/paper/2609.00028/) (2026)
- [STAMP: Training Explicit Memory for Mobile GUI Agents in Controllable and Scalable Virtual Environments](https://mlsanity.com/paper/2605.29324/) (2026)
- [FedGUI: Benchmarking Federated GUI Agents across Heterogeneous Platforms, Devices, and Operating Systems](https://mlsanity.com/paper/2604.14956/) (2026)
- [OS-SPEAR: A Toolkit for the Safety, Performance,Efficiency, and Robustness Analysis of OS Agents](https://mlsanity.com/paper/2604.24348/) (2026)
- [Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability](https://mlsanity.com/paper/2605.07110/) (2026)
- [nanoMuse: An Open-Source Personal Agent for Every Device You Own](https://mlsanity.com/paper/2610.08699/) (2026)
- [DeltaReplay: Task-Relative Memory Reuse for Mobile GUI Agents](https://mlsanity.com/paper/2610.11707/) (2026)

## Abstract (arXiv)

Reliable mobile GUI agents must retain and reuse information across actions, applications, and repeated interactions. However, current benchmarks systematically underrepresent these memory demands: only 5.2-11.8 percent of their tasks are memory-related, and none evaluates cross-session learning. We introduce MemGUI-Bench, a comprehensive memory-centric benchmark that assesses both short-term information retention and long-term experience accumulation through pass@k protocols and staged LLM-as-judge evaluation. Our contributions include: (1) a systematic taxonomy of short- and long-term memory based on 11 agents across 5 architectures; (2) a snapshot-based suite of 128 tasks across 26 applications, organized into 64 mirror pairs, where 89.8 percent require cross-temporal and cross-spatial retention; (3) MemGUI-Eval, an automated 3-stage Progressive Scrutiny pipeline with 7 hierarchical metrics spanning memory fidelity, learning effectiveness, and execution efficiency; and (4) an assessment of 11 state-of-the-art agents guided by 6 research questions. Our experiments reveal substantial memory deficits across all evaluated systems, including 4-10x capability gaps on memory-intensive tasks. They further show that short-term memory is indispensable, while explicit long-term memory improves cross-session learning by 21.9 percentage points, with cross-application transfer and computational cost remaining major bottlenecks. We additionally identify 5 distinct failure modes and synthesize 5 actionable design implications for future memory-enhanced agents. All resources, including code, benchmark, and evaluation results, will be fully open-sourced and continuously maintained at https://memgui-bench.github.io/.