Continual GUI Agents
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
As digital environments (data distribution) are in flux, with new GUI data arriving over time-introducing new domains or resolutions-agents trained on static environments deteriorate in performance. In this work, we introduce Continual GUI Agents, a new task that requires GUI agents to perform continual learning under shifted domains and resolutions. We find existing methods fail to maintain stable grounding as GUI distributions shift over time, due to the diversity of UI interaction points and regions in fluxing scenarios. To address this, we introduce GUI-Anchoring in Flux (GUI-AiF), a new reinforcement fine-tuning framework that stabilizes continual learning through two novel rewards: Anchoring Point Reward in Flux (APR-iF) and Anchoring Region Reward in Flux (ARR-iF). These rewards guide the agents to align with shifting interaction points and regions, mitigating the tendency of existing reward strategies to over-adapt to static grounding cues (e.g., fixed coordinates or element scales). Extensive experiments show GUI-AiF surpasses state-of-the-art baselines. Our work establishes the first continual learning framework for GUI agents, revealing the untapped potential of reinforcement fine-tuning for continual GUI Agents.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 39 of its 47 references that are in the map. First posted on arXiv 28 Jan 2026.
Builds on
39 of 47 references.
Closest ancestors · 15
Same direction, newest first
- 2025How Auxiliary Reasoning Unleashes GUI Grounding in VLMs15 Sep
- 2025KG-RAG: Enhancing GUI Agent Decision-Making via Knowledge Graph-Driven Retrieval-Augmented Generation30 Aug
- 2025Test-Time Reinforcement Learning for GUI Grounding via Region Consistency7 Aug
- 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents3 Jun
- 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning18 May
- 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners19 Apr
- 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents14 Apr
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
- 2024OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web27 Feb
3 more
Other references · 24
- 2023GPT-4 Technical Reportcited by 18,132
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
- 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
- 2023AutoDroid: LLM-powered Task Automation in Androidcited by 178
- 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
- 2024UFO: A UI-Focused Agent for Windows OS Interactioncited by 147
- 2022Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillationcited by 72
- 2025GUI-G2: Gaussian Reward Modeling for GUI Groundingcited by 60
12 more
- 2025Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Trainingcited by 42
- 2024Make Continual Learning Stronger via C-Flatcited by 35
- 2025RL Fine-Tuning Heals OOD Forgetting in SFTcited by 27
- 2025Think Twice, Click Once: Enhancing GUI Grounding via Fast and Slow Systemscited by 23
- 2025Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspectivecited by 14
- 2025LLaVA-c: Continual Improved Visual Instruction Tuningcited by 14
- 2025Why Reinforcement Fine-Tuning Enables MLLMs Preserve Prior Knowledge Better: A Data Perspectivecited by 14
- 2025Learning GUI Grounding with Spatial Reasoning from Visual Feedbackcited by 12
- 2025GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agentscited by 10
- 2025ZeroFlow: Overcoming Catastrophic Forgetting is Easier than You Thinkcited by 9
- 2025Continual Learning for Generative AI: From LLMs to MLLMs and Beyondcited by 9
- 2025textscGUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Groundingcited by 8
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Autonomous Continual Learning for Environment Adaptation of Computer-Use Agentscited by 4
- 2026Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elementscited by 4
- 2026A Faster Path to Continual Learningcited by 1
- 2026Learn from Weaknesses: Automated Domain Specialization for Small Computer-Use Agentscited by 1
- 2026GUI-AC: Enhancing Continual Learning in GUI Agentscited by 1
- 2026Software Engineering for and with GUI Agentcited by 1
- 2026Continual Learning in Transitioncited by 0
- 2026Software Engineering for and with GUI Agent10 Aug 2026
- 2026Continual Learning in Transition6 Aug 2026
- 2026GUI-AC: Enhancing Continual Learning in GUI Agents9 Jun 2026
- 2026Learn from Weaknesses: Automated Domain Specialization for Small Computer-Use Agents27 May 2026
- 2026A Faster Path to Continual Learning13 Apr 2026
- 2026Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elements15 Mar 2026
- 2026Autonomous Continual Learning for Environment Adaptation of Computer-Use Agents10 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2026CGL: Advancing Continual GUI Learning via Reinforcement Fine-Tuning
- 2026GUI Agents with Reinforcement Learning: Toward Digital Inhabitants
- 2026MagicGUI-RMS: A Multi-Agent Reward Model System for Self-Evolving GUI Agents via Automated Feedback Reflux
- 2025GuirlVG: Incentivize GUI Visual Grounding via Empirical Exploration on Reinforcement Learning
- 2026GUI-C2: Coarse-to-Fine GUI Grounding via Difficulty-Aware Reinforcement Learning
- 2025CRAFT-GUI: Curriculum-Reinforced Agent For GUI Tasks
- 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents
- 2025UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time Grounding