# UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization

arXiv: https://arxiv.org/abs/2604.13822 · HTML: https://mlsanity.com/paper/2604.13822/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use.
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Placed in

- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)

## Builds on (most cited first, 27 of 27)

- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents](https://mlsanity.com/paper/2401.10935/) (2024)
- [AppAgent: Multimodal Agents as Smartphone Users](https://mlsanity.com/paper/2312.13771/) (2023)
- [Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction](https://mlsanity.com/paper/2412.04454/) (2024)
- [ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use](https://mlsanity.com/paper/2504.07981/) (2025)
- [GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents](https://mlsanity.com/paper/2504.10458/) (2025)
- [ShowUI: One Vision-Language-Action Model for GUI Visual Agent](https://mlsanity.com/paper/2411.17465/) (2024)
- [GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devices](https://mlsanity.com/paper/2406.08451/) (2024)
- [UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning](https://mlsanity.com/paper/2503.21620/) (2025)
- [Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents](https://mlsanity.com/paper/2504.00906/) (2025)
- [Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks](https://mlsanity.com/paper/2501.11733/) (2025)
- [SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization](https://mlsanity.com/paper/2604.02268/) (2026)
- [UI-Venus Technical Report: Building High-performance UI Agents with RFT](https://mlsanity.com/paper/2508.10833/) (2025)
- [ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replay](https://mlsanity.com/paper/2505.16282/) (2025)
- [OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use](https://mlsanity.com/paper/2508.04482/) (2025)
- [UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time Grounding](https://mlsanity.com/paper/2507.22025/) (2025)
- [Test-Time Reinforcement Learning for GUI Grounding via Region Consistency](https://mlsanity.com/paper/2508.05615/) (2025)
- [KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation](https://mlsanity.com/paper/2604.08455/) (2026)
- [MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments](https://mlsanity.com/paper/2602.06075/) (2026)
- [Retrieval-augmented GUI Agents with Generative Guidelines](https://mlsanity.com/paper/2509.24183/) (2025)
- [EchoTrail-GUI: Building Actionable Memory for GUI Agents via Critic-Guided Self-Exploration](https://mlsanity.com/paper/2512.19396/) (2025)
- [HiconAgent: History Context-aware Policy Optimization for GUI Agents](https://mlsanity.com/paper/2512.01763/) (2025)
- [PAL-UI: Planning with Active Look-back for Vision-Based GUI Agents](https://mlsanity.com/paper/2510.00413/) (2025)
- [History-Aware Reasoning for GUI Agents](https://mlsanity.com/paper/2511.09127/) (2025)
- [Spark: Strategic Policy-Aware Exploration via Dynamic Branching for Long-Horizon Agentic Learning](https://mlsanity.com/paper/2601.20209/) (2026)
- [NaviMaster: Learning a Unified Policy for GUI and Embodied Navigation Tasks](https://mlsanity.com/paper/2508.02046/) (2025)
- [GUI-Rise: Structured Reasoning and History Summarization for GUI Navigation](https://mlsanity.com/paper/2510.27210/) (2025)

## Built on by (most cited first, 4 of 4)

- [Self-Distilled Agentic Reinforcement Learning](https://mlsanity.com/paper/2605.15155/) (2026)
- [A Task-State Representation for Long-Horizon Mobile GUI Agents](https://mlsanity.com/paper/2607.00502/) (2026)
- [BrowserForge: Scaling Web Episode via Parallel Browser Sandboxes](https://mlsanity.com/paper/2608.24848/) (2026)
- [HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents](https://mlsanity.com/paper/2609.38008/) (2026)

## Abstract (arXiv)

MLLM-based GUI agents have demonstrated strong capabilities in complex user interface interaction tasks. However, long-horizon scenarios remain challenging, as these agents are burdened with tasks beyond their intrinsic capabilities, suffering from memory degradation, progress confusion, and math hallucination. To address these challenges, we present UI-Copilot, a collaborative framework where the GUI agent focuses on task execution while a lightweight copilot provides on-demand assistance for memory retrieval and numerical computation. We introduce memory decoupling to separate persistent observations from transient execution context, and train the policy agent to selectively invoke the copilot as Retriever or Calculator based on task demands. To enable effective tool invocation learning, we propose Tool-Integrated Policy Optimization (TIPO), which separately optimizes tool selection through single-turn prediction and task execution through on-policy multi-turn rollouts. Experimental results show that UI-Copilot-7B achieves state-of-the-art performance on challenging MemGUI-Bench, outperforming strong 7B-scale GUI agents such as GUI-Owl-7B and UI-TARS-1.5-7B. Moreover, UI-Copilot-7B delivers a 17.1% absolute improvement on AndroidWorld over the base Qwen model, highlighting UI-Copilot's strong generalization to real-world GUI tasks.