mlsanityData to 7 Oct 2026

EchoTrail-GUI: Building Actionable Memory for GUI Agents via Critic-Guided Self-Exploration

arXiv 2512.19396 ↗First posted 22 Dec 2025cs.AIcited by 13

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Contemporary GUI agents, while increasingly capable due to advances in Large Vision-Language Models (VLMs), often operate with a critical limitation: they treat each task in isolation, lacking a mechanism to systematically learn from past successes. This digital ''amnesia'' results in sub-optimal performance, repeated errors, and poor generalization to novel challenges. To bridge this gap, we introduce EchoTrail-GUI, a novel framework designed to mimic human-like experiential learning by equipping agents with a dynamic, accessible memory. Our framework operates in three distinct stages. First, during Experience Exploration, an agent autonomously interacts with GUI environments to build a curated database of successful task trajectories, validated by a reward model. Crucially, the entire knowledge base construction is thus fully automated, requiring no human supervision. Second, in the Memory Injection stage, upon receiving a new task, our system efficiently retrieves the most relevant past trajectories to serve as actionable ''memories''. Finally, during GUI Task Inference, these memories are injected as in-context guidance to inform the agent's reasoning and decision-making process. We demonstrate the efficacy of our approach on benchmarks including Android World and AndroidLab. The results show that EchoTrail-GUI significantly improves the task success rate and operational efficiency of baseline agents, validating the power of structured memory in creating more robust and intelligent GUI automation.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 37 references that are in the map. First posted on arXiv 22 Dec 2025.

Builds on

32 of 37 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Retrieval-augmented GUI Agents with Generative Guidelines29 Sep
  2. 2025Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error Diagnosis in GUI Automation5 Jun
  3. 2025Scalable Video-to-Dataset Generation for Cross-Platform Mobile Agents19 May
  4. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  5. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
  6. 2023CogAgent: A Visual Language Model for GUI Agents14 Dec

Other references · 26

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  8. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  9. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  10. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  11. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  12. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
14 more
  1. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
  2. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
  3. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 187
  4. 2023AutoDroid: LLM-powered Task Automation in Androidcited by 178
  5. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 155
  6. 2024Autonomous Evaluation and Refinement of Digital Agentscited by 136
  7. 2024Aria-UI: Visual Grounding for GUI Instructionscited by 136
  8. 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
  9. 2024AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agentscited by 105
  10. 2024AutoGLM: Autonomous Foundation Agents for GUIscited by 96
  11. 2025GUI-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for GUI Agentcited by 35
  12. 2024Ponder & Press: Advancing Visual GUI Agent towards General Computer Controlcited by 19
  13. 2025ScaleTrack: Scaling and back-tracking Automated GUI Agentscited by 16
  14. 2025AndroidGen: Building an Android Language Agent under Data Scarcitycited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2025MGA: Memory-Driven GUI Agent for Observation-Centric Interaction
  2. 2026ActionEngine: From Reactive to Programmatic GUI Agents via State Machine Memory
  3. 2026STAMP: Training Explicit Memory for Mobile GUI Agents in Controllable and Scalable Virtual Environments
  4. 2026AnchorGUI: Asymmetric Memory for Dual-Scale Learning in GUI Navigation
  5. 2026Learning Reliable GUI Agents under Imperfect Priors
  6. 2025Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks
  7. 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
  8. 2025GUI Exploration Lab: Enhancing Screen Navigation in Agents via Multi-Turn Reinforcement Learning