EchoTrail-GUI: Building Actionable Memory for GUI Agents via Critic-Guided Self-Exploration
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
Contemporary GUI agents, while increasingly capable due to advances in Large Vision-Language Models (VLMs), often operate with a critical limitation: they treat each task in isolation, lacking a mechanism to systematically learn from past successes. This digital ''amnesia'' results in sub-optimal performance, repeated errors, and poor generalization to novel challenges. To bridge this gap, we introduce EchoTrail-GUI, a novel framework designed to mimic human-like experiential learning by equipping agents with a dynamic, accessible memory. Our framework operates in three distinct stages. First, during Experience Exploration, an agent autonomously interacts with GUI environments to build a curated database of successful task trajectories, validated by a reward model. Crucially, the entire knowledge base construction is thus fully automated, requiring no human supervision. Second, in the Memory Injection stage, upon receiving a new task, our system efficiently retrieves the most relevant past trajectories to serve as actionable ''memories''. Finally, during GUI Task Inference, these memories are injected as in-context guidance to inform the agent's reasoning and decision-making process. We demonstrate the efficacy of our approach on benchmarks including Android World and AndroidLab. The results show that EchoTrail-GUI significantly improves the task success rate and operational efficiency of baseline agents, validating the power of structured memory in creating more robust and intelligent GUI automation.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 37 references that are in the map. First posted on arXiv 22 Dec 2025.
Builds on
32 of 37 references.
Closest ancestors · 6
Same direction, newest first
- 2025Retrieval-augmented GUI Agents with Generative Guidelines29 Sep
- 2025Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error Diagnosis in GUI Automation5 Jun
- 2025Scalable Video-to-Dataset Generation for Cross-Platform Mobile Agents19 May
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
- 2023CogAgent: A Visual Language Model for GUI Agents14 Dec
Other references · 26
- 2023GPT-4 Technical Reportcited by 18,132
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
- 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
14 more
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
- 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
- 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 187
- 2023AutoDroid: LLM-powered Task Automation in Androidcited by 178
- 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 155
- 2024Autonomous Evaluation and Refinement of Digital Agentscited by 136
- 2024Aria-UI: Visual Grounding for GUI Instructionscited by 136
- 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
- 2024AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agentscited by 105
- 2024AutoGLM: Autonomous Foundation Agents for GUIscited by 96
- 2025GUI-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for GUI Agentcited by 35
- 2024Ponder & Press: Advancing Visual GUI Agent towards General Computer Controlcited by 19
- 2025ScaleTrack: Scaling and back-tracking Automated GUI Agentscited by 16
- 2025AndroidGen: Building an Android Language Agent under Data Scarcitycited by 10
Built on by
Citing papers per year: 2026: 13.
13 papers.
Sort:
By area:
- 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 71
- 2026MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agentscited by 9
- 2026Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolutioncited by 7
- 2026Enhancing Web Agents with a Hierarchical Memory Treecited by 5
- 2026What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving Statescited by 5
- 2026RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environmentscited by 5
- 2026UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimizationcited by 4
- 2026Executable Agentic Memory for GUI Agentcited by 4
- 2026StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structurecited by 4
- 2026Mem-W: Latent Memory-Native GUI Agentscited by 3
- 2026FocusMem: Factorizing Content, Readout, and Trust in Latent GUI Memorycited by 2
- 2026EchoPath: Execution-Level Replayable Memory for GUI Agentscited by 1
1 more
- 2026One Rollout Is All You Get: Fully Test-Time Adaptation for GUI Agents28 Sep 2026
- 2026EchoPath: Execution-Level Replayable Memory for GUI Agents15 Sep 2026
- 2026RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments14 Sep 2026
- 2026FocusMem: Factorizing Content, Readout, and Trust in Latent GUI Memory5 Aug 2026
- 2026StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure13 Jul 2026
- 2026What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving States30 Jun 2026
- 2026MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents18 May 2026
- 2026Executable Agentic Memory for GUI Agent12 May 2026
- 2026Mem-W: Latent Memory-Native GUI Agents10 May 2026
- 2026UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization15 Apr 2026
- 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agents12 Mar 2026
- 2026Enhancing Web Agents with a Hierarchical Memory Tree7 Mar 2026
- 2026MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agentscited by 9
- 2026Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolutioncited by 7
- 2026Enhancing Web Agents with a Hierarchical Memory Treecited by 5
- 2026What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving Statescited by 5
- 2026UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimizationcited by 4
- 2026Executable Agentic Memory for GUI Agentcited by 4
- 2026Mem-W: Latent Memory-Native GUI Agentscited by 3
- 2026FocusMem: Factorizing Content, Readout, and Trust in Latent GUI Memorycited by 2
- 2026EchoPath: Execution-Level Replayable Memory for GUI Agentscited by 1
- 2026One Rollout Is All You Get: Fully Test-Time Adaptation for GUI Agentscited by 0
- 2026One Rollout Is All You Get: Fully Test-Time Adaptation for GUI Agents28 Sep 2026
- 2026EchoPath: Execution-Level Replayable Memory for GUI Agents15 Sep 2026
- 2026FocusMem: Factorizing Content, Readout, and Trust in Latent GUI Memory5 Aug 2026
- 2026What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving States30 Jun 2026
- 2026MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents18 May 2026
- 2026Executable Agentic Memory for GUI Agent12 May 2026
- 2026Mem-W: Latent Memory-Native GUI Agents10 May 2026
- 2026UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization15 Apr 2026
- 2026Enhancing Web Agents with a Hierarchical Memory Tree7 Mar 2026
- 2026Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolution30 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025MGA: Memory-Driven GUI Agent for Observation-Centric Interaction
- 2026ActionEngine: From Reactive to Programmatic GUI Agents via State Machine Memory
- 2026STAMP: Training Explicit Memory for Mobile GUI Agents in Controllable and Scalable Virtual Environments
- 2026AnchorGUI: Asymmetric Memory for Dual-Scale Learning in GUI Navigation
- 2026Learning Reliable GUI Agents under Imperfect Priors
- 2025Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks
- 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
- 2025GUI Exploration Lab: Enhancing Screen Navigation in Agents via Multi-Turn Reinforcement Learning