mlsanityData to 14 Oct 2026

Reflect, Revise, Reuse: Training-Free Skill Evolution for GUI Agents

arXiv 2609.17653 ↗First posted 15 Sep 2026cs.LG cs.AIcited by 2

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

GUI agents execute long-horizon tasks on dynamic graphical user interfaces, where pop-ups, delayed loads, and relocated widgets routinely invalidate plans fixed before execution. Recent agent-skill frameworks encapsulate reusable procedural knowledge to mitigate this, yet existing skill designs are largely developed without targeting GUI execution dynamics and treat skills as static artifacts produced before deployment rather than living procedural knowledge that improves through it. We argue that what GUI agents need is not better static skills, but skills that can be revised from execution feedback at deployment time, without additional training. We propose EvoSkill-GUI, a training-free framework in which each skill is a structured multi-file package containing retrieval metadata, executable plans, backup localization, failure-recovery rules, accessibility utilities, and failure cases. EvoSkill-GUI operates through a textbfreflect-revise-reuse loop: the executor performs instant in-rollout revisions, an isolated critic diagnoses failed trajectories under strict information isolation, and the executor edits specific skill files through a restricted tool interface. Across MobileWorld, AndroidWorld, and OSWorld, three mainstream GUI benchmarks spanning mobile and desktop platforms, EvoSkill-GUI consistently improves multiple base models without any training, with maximum gains of +16.2%, +6.0%, and +10.5% respectively, and evolved skill libraries continue to benefit related tasks rather than being rebuilt from scratch. Our code is available at https://github.com/ZJU-REAL/EvoSkill-GUI.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 40 of its 44 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

40 of 44 references.

Closest ancestors · 17

Same direction, newest first

  1. 2026Don't Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-Correction7 Apr
  2. 2026UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experience25 Mar
  3. 2026UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agents5 Feb
  4. 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agents26 Dec
  5. 2025MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments22 Dec
  6. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automation21 Aug
  7. 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
  8. 2025MobileUse: A GUI Agent with Hierarchical Reflection for Autonomous Mobile Operation21 Jul
  9. 2025UI-Evol: Automatic Knowledge Evolving for Computer Use Agents28 May
  10. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
  11. 2025A Survey on (M)LLM-Based GUI Agents27 Mar
  12. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning27 Mar
5 more
  1. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
  2. 2025Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks20 Jan
  3. 2024GUI Agents: A Survey18 Dec
  4. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr
  5. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents17 Jan

Other references · 23

  1. 2025Qwen3-VL Technical Reportcited by 2,793
  2. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  3. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 278
  4. 2026CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verificationcited by 128
  5. 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 113
  6. 2025Memp: Exploring Agent Procedural Memorycited by 103
  7. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalizationcited by 94
  8. 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 75
  9. 2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experiencecited by 69
  10. 2025GUI-G2: Gaussian Reward Modeling for GUI Groundingcited by 63
  11. 2025WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Modelcited by 61
  12. 2026Self-Distilled Agentic Reinforcement Learningcited by 55
11 more
  1. 2026EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experiencecited by 53
  2. 2026Dynamic Dual-Granularity Skill Bank for Agentic RLcited by 34
  3. 2026CUA-Skill: Develop Skills for Computer Using Agentcited by 33
  4. 2026WebXSkill: Skill Learning for Autonomous Web Agentscited by 24
  5. 2025GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection Behaviorcited by 23
  6. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 21
  7. 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applicationscited by 18
  8. 2026MMSkills: Towards Multimodal Skills for General Visual Agentscited by 16
  9. 2026Hybrid Self-evolving Structured Memory for GUI Agentscited by 13
  10. 2026Experiential Reflective Learning for Self-Improving LLM Agentscited by 13
  11. 2026HyMem: Hybrid Memory Architecture with Dynamic Retrieval Schedulingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction
  2. 2026From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use Agents
  3. 2026GUI vs. CLI: Execution Bottlenecks in Screen-Only and Skill-Mediated Computer-Use Agents
  4. 2026SkillDroid: Compile Once, Reuse Forever
  5. 2026ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents
  6. 2026Learning Reliable GUI Agents under Imperfect Priors
  7. 2026GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training
  8. 2026Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization