# Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents

arXiv: https://arxiv.org/abs/2607.28227 · HTML: https://mlsanity.com/paper/2607.28227/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use.
- Builds on 48 of its 79 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Placed in

- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)

## Builds on (most cited first, 30 of 48)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [A Diagram Is Worth A Dozen Images](https://mlsanity.com/paper/1603.07396/) (2016)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [GPT-4V(ision) is a Generalist Web Agent, if Grounded](https://mlsanity.com/paper/2401.01614/) (2024)
- [Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset](https://mlsanity.com/paper/2402.14804/) (2024)
- [MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark](https://mlsanity.com/paper/2409.02813/) (2024)
- [AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents](https://mlsanity.com/paper/2405.14573/) (2024)
- [WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models](https://mlsanity.com/paper/2401.13919/) (2024)
- [OS-ATLAS: A Foundation Action Model for Generalist GUI Agents](https://mlsanity.com/paper/2410.23218/) (2024)
- [SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks](https://mlsanity.com/paper/2602.12670/) (2026)
- [CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs](https://mlsanity.com/paper/2406.18521/) (2024)
- [UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2509.02544/) (2025)
- [An Illusion of Progress? Assessing the Current State of Web Agents](https://mlsanity.com/paper/2504.01382/) (2025)
- [OpenCUA: Open Foundations for Computer-Use Agents](https://mlsanity.com/paper/2508.09123/) (2025)
- [BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese](https://mlsanity.com/paper/2504.19314/) (2025)
- [Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis](https://mlsanity.com/paper/2505.13227/) (2025)
- [GTA1: GUI Test-time Scaling Agent](https://mlsanity.com/paper/2507.05791/) (2025)
- [MAI-UI Technical Report: Real-World Centric Foundation GUI Agents](https://mlsanity.com/paper/2512.22047/) (2025)
- [Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents](https://mlsanity.com/paper/2602.16855/) (2026)
- [UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction](https://mlsanity.com/paper/2503.15661/) (2025)
- [MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents](https://mlsanity.com/paper/2507.19478/) (2025)
- [Tongyi DeepResearch Technical Report](https://mlsanity.com/paper/2510.24701/) (2025)
- [Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents](https://mlsanity.com/paper/2604.06132/) (2026)
- [UI-Venus-1.5 Technical Report](https://mlsanity.com/paper/2602.09082/) (2026)

## Built on by (most cited first, 11 of 11)

- [UI-Venus-2 Technical Report](https://mlsanity.com/paper/2609.00028/) (2026)
- [UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations](https://mlsanity.com/paper/2608.15930/) (2026)
- [Wuying-Browser-Agent: Real-World Centric Fundamental Long-Horizon Browser Agents](https://mlsanity.com/paper/2608.17319/) (2026)
- [RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents](https://mlsanity.com/paper/2609.22000/) (2026)
- [From Tapping to Hopping: Augmenting Mobile GUI Agents with App-Native Deeplinks](https://mlsanity.com/paper/2609.30887/) (2026)
- [CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learning](https://mlsanity.com/paper/2609.32750/) (2026)
- [Org-Agent: Beyond Personal Assistants Towards Organizational Agents](https://mlsanity.com/paper/2609.34392/) (2026)
- [HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents](https://mlsanity.com/paper/2609.38008/) (2026)
- [CUAWright: A Minimal Unified Interface for Digital Agents](https://mlsanity.com/paper/2610.04116/) (2026)
- [PlaySuite: A Large-Scale Benchmark for Interactive Visual Intelligence](https://mlsanity.com/paper/2610.07127/) (2026)
- [OpenMobile-2: Building Versatile Mobile Agents with Scalable Environments and App-Native Tools](https://mlsanity.com/paper/2610.14157/) (2026)

## Abstract (arXiv)

GUI agents have the potential to become a general purpose executor over existing digital devices. To advance them toward real-world use, we envision agents that operate reliably on real devices, execute workflows across platforms, combine GUI interaction with CLI execution, complete long-horizon tasks, proactively initiate useful services, and autonomously improve their capabilities with minimal human effort. Guided by this vision, we present Qwen-UI-Agent, a real-world centric foundation GUI agent spanning mobile, computer-use, web, and DeepSearch environments. Qwen-UI-Agent combines diverse sandbox environments with a large-scale real-device mobile runtime. Its unified action space interleaves GUI operations with CLI execution and generates batched actions in a single model turn. An AutoResearch-style data flywheel uses agents to construct tasks and environments, diagnose failures, and plan subsequent iterations. Online RL supports training on trajectories exceeding 100 turns, with over 10,000 concurrent environments accelerating rollout. A lightweight harness layer supports proactive service initiation and stateful workflows across mobile and computer. Across a broad suite of evaluations, Qwen-UI-Agent sets state-of-the-art performance on mobile-use benchmarks while delivering competitive performance on computer- and browser-use tasks against frontier models, including Opus 4.8, Gemini 3.1 Pro, and GPT-5.6 Sol. On mobile use, it achieves 82.1% on MobileWorld, 92.2% on MobileWorld-Real, and 97.5% on AndroidDaily. On computer use, it achieves 79.5% on OSWorld-Verified and a 40.0% partial-progress score on OSWorld-v2. On browser use and GUI grounding, it achieves 73.6% on WebArena and 81.5% on ScreenSpot-Pro, respectively.