mlsanityData to 7 Oct 2026

Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents

arXiv 2607.28227 ↗First posted 30 Jul 2026cs.AI cs.CVcited by 8

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

GUI agents have the potential to become a general purpose executor over existing digital devices. To advance them toward real-world use, we envision agents that operate reliably on real devices, execute workflows across platforms, combine GUI interaction with CLI execution, complete long-horizon tasks, proactively initiate useful services, and autonomously improve their capabilities with minimal human effort. Guided by this vision, we present Qwen-UI-Agent, a real-world centric foundation GUI agent spanning mobile, computer-use, web, and DeepSearch environments. Qwen-UI-Agent combines diverse sandbox environments with a large-scale real-device mobile runtime. Its unified action space interleaves GUI operations with CLI execution and generates batched actions in a single model turn. An AutoResearch-style data flywheel uses agents to construct tasks and environments, diagnose failures, and plan subsequent iterations. Online RL supports training on trajectories exceeding 100 turns, with over 10,000 concurrent environments accelerating rollout. A lightweight harness layer supports proactive service initiation and stateful workflows across mobile and computer. Across a broad suite of evaluations, Qwen-UI-Agent sets state-of-the-art performance on mobile-use benchmarks while delivering competitive performance on computer- and browser-use tasks against frontier models, including Opus 4.8, Gemini 3.1 Pro, and GPT-5.6 Sol. On mobile use, it achieves 82.1% on MobileWorld, 92.2% on MobileWorld-Real, and 97.5% on AndroidDaily. On computer use, it achieves 79.5% on OSWorld-Verified and a 40.0% partial-progress score on OSWorld-v2. On browser use and GUI grounding, it achieves 73.6% on WebArena and 81.5% on ScreenSpot-Pro, respectively.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 48 of its 79 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

48 of 79 references.

Closest ancestors · 20

Same direction, newest first

  1. 2026HyMobileAgent: Data-Environment Co-Scaling for Efficient GUI Agents16 Jul
  2. 2026UI-MOPD: Multi-Platform On-Policy Distillation for Unified GUI Agents5 Jul
  3. 2026Xiaomi-GUI-0 Technical Report30 Jun
  4. 2026OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks28 Jun
  5. 2026PhoneBuddy: Training Open Models for Agentic Phone Use22 Jun
  6. 2026AndroidDaily: A Verifiable Benchmark for Mobile GUI Agents on Real-World Closed-Source Applications26 May
  7. 2026ToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agents12 May
  8. 2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents15 Feb
  9. 2026UI-Venus-1.5 Technical Report9 Feb
  10. 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agents26 Dec
  11. 2025Step-GUI Technical Report17 Dec
  12. 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoning23 Oct
8 more
  1. 2025UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action20 Oct
  2. 2025Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation28 Sep
  3. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
  4. 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
  5. 2025MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents25 Jul
  6. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis19 May
  7. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
  8. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct

Other references · 28

  1. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  2. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  3. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  4. 2016A Diagram Is Worth A Dozen Imagescited by 793
  5. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  6. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  7. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  8. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
  9. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  10. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  11. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  12. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
16 more
  1. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  2. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
  3. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 229
  4. 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 165
  5. 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 139
  6. 2025GTA1: GUI Test-time Scaling Agentcited by 102
  7. 2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interactioncited by 68
  8. 2026Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agentscited by 57
  9. 2025Tongyi DeepResearch Technical Reportcited by 53
  10. 2026Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learningcited by 26
  11. 2026WebWorld: A Large-Scale World Model for Web Agent Trainingcited by 23
  12. 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agentscited by 21
  13. 2026Odysseys: Benchmarking Web Agents on Realistic Long Horizon Taskscited by 19
  14. 2026PhoneHarness: Harnessing Phone-Use Agents through Mixed GUI, CLI, and Tool Actionscited by 6
  15. 2026EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agentscited by 2
  16. 2026One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Groundingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Mobile GUI Agents under Real-world Threats: Are We There Yet?
  2. 2026APPSim-Bench: Bridging Real-world Apps and Reproducible Evaluation for Mobile GUI Agents
  3. 2026ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents
  4. 2026CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents
  5. 2026BlueLM-GUI Technical Report: A Real-Device-Centric Flywheel for Self-Improving Mobile GUI Agents
  6. 2026VenusBench-Mobile: A Challenging and User-Centric Benchmark for Mobile GUI Agents with Capability Diagnostics
  7. 2026WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
  8. 2026ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis