mlsanityData to 7 Oct 2026

UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization

arXiv 2604.13822 ↗First posted 15 Apr 2026cs.LGcited by 4

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

MLLM-based GUI agents have demonstrated strong capabilities in complex user interface interaction tasks. However, long-horizon scenarios remain challenging, as these agents are burdened with tasks beyond their intrinsic capabilities, suffering from memory degradation, progress confusion, and math hallucination. To address these challenges, we present UI-Copilot, a collaborative framework where the GUI agent focuses on task execution while a lightweight copilot provides on-demand assistance for memory retrieval and numerical computation. We introduce memory decoupling to separate persistent observations from transient execution context, and train the policy agent to selectively invoke the copilot as Retriever or Calculator based on task demands. To enable effective tool invocation learning, we propose Tool-Integrated Policy Optimization (TIPO), which separately optimizes tool selection through single-turn prediction and task execution through on-policy multi-turn rollouts. Experimental results show that UI-Copilot-7B achieves state-of-the-art performance on challenging MemGUI-Bench, outperforming strong 7B-scale GUI agents such as GUI-Owl-7B and UI-TARS-1.5-7B. Moreover, UI-Copilot-7B delivers a 17.1% absolute improvement on AndroidWorld over the base Qwen model, highlighting UI-Copilot's strong generalization to real-world GUI tasks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 27 of its 39 references that are in the map. First posted on arXiv 15 Apr 2026.

Builds on

27 of 39 references.

Closest ancestors · 17

Same direction, newest first

  1. 2026KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation9 Apr
  2. 2025EchoTrail-GUI: Building Actionable Memory for GUI Agents via Critic-Guided Self-Exploration22 Dec
  3. 2025HiconAgent: History Context-aware Policy Optimization for GUI Agents1 Dec
  4. 2025History-Aware Reasoning for GUI Agents12 Nov
  5. 2025PAL-UI: Planning with Active Look-back for Vision-Based GUI Agents1 Oct
  6. 2025UI-Venus Technical Report: Building High-performance UI Agents with RFT14 Aug
  7. 2025Test-Time Reinforcement Learning for GUI Grounding via Region Consistency7 Aug
  8. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use6 Aug
  9. 2025UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time Grounding29 Jul
  10. 2025ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replay22 May
  11. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents14 Apr
  12. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
5 more
  1. 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents1 Apr
  2. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning27 Mar
  3. 2025Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks20 Jan
  4. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents17 Jan
  5. 2023AppAgent: Multimodal Agents as Smartphone Users21 Dec

Other references · 10

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  2. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
  3. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
  4. 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 190
  5. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalizationcited by 82
  6. 2026MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environmentscited by 17
  7. 2025Retrieval-augmented GUI Agents with Generative Guidelinescited by 14
  8. 2025NaviMaster: Learning a Unified Policy for GUI and Embodied Navigation Taskscited by 10
  9. 2026Spark: Strategic Policy-Aware Exploration via Dynamic Branching for Long-Horizon Agentic Learningcited by 9
  10. 2025GUI-Rise: Structured Reasoning and History Summarization for GUI Navigationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026ToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agents
  2. 2025MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents
  3. 2026LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning
  4. 2026Beyond Sequential Interaction: Benchmarking Parallel Execution and Coordination for GUI Agents
  5. 2025WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point
  6. 2026ActionEngine: From Reactive to Programmatic GUI Agents via State Machine Memory
  7. 2026Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization
  8. 2025Chain-of-Memory: Enhancing GUI Agents for Cross-Application Navigation