AndroTMem: From Interaction Trajectories to Anchored Memory in Long-Horizon GUI Agents
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
Long-horizon GUI agents are a key step toward real-world deployment, yet effective interaction memory under prevailing paradigms remains under-explored. Replaying full interaction sequences is redundant and amplifies noise, while summaries often erase dependency-critical information and traceability. We present AndroTMem, a diagnostic framework for anchored memory in long-horizon Android GUI agents. Its core benchmark, AndroTMem-Bench, comprises 1,069 tasks with 34,473 interaction steps (avg. 32.1 per task, max. 65). We evaluate agents with TCR (Task Complete Rate), focusing on tasks whose completion requires carrying forward critical intermediate state; AndroTMem-Bench is designed to enforce strong step-to-step causal dependencies, making sparse yet essential intermediate states decisive for downstream actions and centering interaction memory in evaluation. Across open- and closed-source GUI agents, we observe a consistent pattern: as interaction sequences grow longer, performance drops are driven mainly by within-task memory failures, not isolated perception errors or local action mistakes. Guided by this diagnosis, we propose Anchored State Memory (ASM), which represents interaction sequences as a compact set of causally linked intermediate-state anchors to enable subgoal-targeted retrieval and attribution-aware decision making. Across multiple settings and 12 evaluated GUI agents, ASM consistently outperforms full-sequence replay and summary-based baselines, improving TCR by 5%-30.16% and AMS by 4.93%-24.66%, indicating that anchored, structured memory effectively mitigates the interaction-memory bottleneck in long-horizon GUI tasks. The code, benchmark, and related resources are publicly available at [https://github.com/CVC2233/AndroTMem](https://github.com/CVC2233/AndroTMem).
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 51 of its 58 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
51 of 58 references.
Closest ancestors · 26
Same direction, newest first
- 2025MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments22 Dec
- 2025UI-Venus Technical Report: Building High-performance UI Agents with RFT14 Aug
- 2025MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents25 Jul
- 2025Mobile-R1: Towards Interactive Capability for VLM-Based Mobile Agent via Systematic Training25 Jun
- 2025AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning2 Jun
- 2025Mobile-Bench-v2: A More Realistic and Comprehensive Benchmark for VLM-based Mobile Agents17 May
- 2025A Survey on GUI Agents with Foundation Models Enhanced by Reinforcement Learning29 Apr
- 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners19 Apr
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
- 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning27 Mar
- 2025COLA: A Scalable Multi-Agent Framework For Windows UI Task Automation12 Mar
- 2025PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PC20 Feb
14 more
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
- 2024Large Language Model-Brained GUI Agents: A Survey27 Nov
- 2024GUI Agents with Foundation Models: A Comprehensive Survey7 Nov
- 2024AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents31 Oct
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
- 2024SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation19 Oct
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
- 2024AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agents3 Jul
- 2024MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agents12 Jun
- 2024On the Effects of Data Scale on UI Control Agents6 Jun
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr
- 2024Android in the Zoo: Chain-of-Action-Thought for GUI Agents5 Mar
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents17 Jan
- 2023CogAgent: A Visual Language Model for GUI Agents14 Dec
Other references · 25
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2025OpenAI GPT-5 System Cardcited by 910
- 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
- 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
- 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
- 2023Video Understanding with Large Language Models: A Surveycited by 203
13 more
- 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 190
- 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Modelscited by 112
- 2025AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learningcited by 68
- 2026OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Modelscited by 23
- 2025RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Videocited by 21
- 2024Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarioscited by 19
- 2025VideoMark: A Distortion-Free Robust Watermarking Framework for Video Diffusion Modelscited by 16
- 2025JavisGPT: A Unified Multi-modal LLM for Sounding-Video Comprehension and Generationcited by 12
- 2025HyperG: Hypergraph-Enhanced LLMs for Structured Knowledgecited by 11
- 2025Mind the Third Eye! Benchmarking Privacy Awareness in MLLM-powered Smartphone Agentscited by 8
- 2025MOSS-ChatV: Reinforcement Learning with Process Reasoning Reward for Video Temporal Reasoningcited by 7
- 2026A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Modelcited by 3
- 2025KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialoguescited by 2
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolutioncited by 12
- 2026What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving Statescited by 5
- 2026What's Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI Reasoningcited by 4
- 2026Mobile GUI Agent Privacy Personalization with Trajectory Induced Preference Optimizationcited by 2
- 2026Rubric-Guided Process Reward for Stepwise Model Routingcited by 2
- 2026Recovering Policy-Induced Errors: Benchmarking and Trajectory Synthesis for Robust GUI Agentscited by 2
- 2026FocusMem: Factorizing Content, Readout, and Trust in Latent GUI Memorycited by 2
- 2026VistaHop: Benchmarking Long-Horizon Visual DeepSearchcited by 1
- 2026Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliabilitycited by 0
- 2026CausalCache: Conditional High-Fidelity Restoration for Long-Horizon GUI Agentscited by 0
- 2026CausalCache: Conditional High-Fidelity Restoration for Long-Horizon GUI Agents23 Aug 2026
- 2026FocusMem: Factorizing Content, Readout, and Trust in Latent GUI Memory5 Aug 2026
- 2026What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving States30 Jun 2026
- 2026VistaHop: Benchmarking Long-Horizon Visual DeepSearch2 Jun 2026
- 2026Recovering Policy-Induced Errors: Benchmarking and Trajectory Synthesis for Robust GUI Agents28 May 2026
- 2026Rubric-Guided Process Reward for Stepwise Model Routing28 May 2026
- 2026Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability8 May 2026
- 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolution29 Apr 2026
- 2026Mobile GUI Agent Privacy Personalization with Trajectory Induced Preference Optimization13 Apr 2026
- 2026What's Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI Reasoning8 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2025Chain-of-Memory: Enhancing GUI Agents for Cross-Application Navigation
- 2026Executable Agentic Memory for GUI Agent
- 2025AgentProg: Empowering Long-Horizon GUI Agents with Program-Guided Context Management
- 2026MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge Evolution
- 2026Mem-W: Latent Memory-Native GUI Agents
- 2025AndroidLens: Long-latency Evaluation with Nested Sub-targets for Android GUI Agents
- 2026A Task-State Representation for Long-Horizon Mobile GUI Agents
- 2026SE-GA: Memory-Augmented Self-Evolution for GUI Agents