Xiaomi-GUI-0 Technical Report
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
Graphical user interface (GUI) agents build on vision-language models to complete user tasks end-to-end in real applications through interface actions such as tapping, swiping, text entry, and navigation. However, existing GUI agents are trained and evaluated largely on offline trajectories, simulated environments, and standardized benchmarks. These differ substantially from real applications in interface layout, interaction logic, and abnormal-state distribution, and cannot faithfully characterize execution stability in real-world use, where account states, permission dialogs, payment authentication, and risk control continually reshape the state distribution and open a persistent gap between benchmark scores and real usability. To close this gap, we propose Xiaomi-GUI-0, a native multimodal GUI agent for real mobile environments, trained and evaluated within a real-device closed loop. At its core is a real-device-dominant hybrid infrastructure, where physical devices are the primary execution environment and sandboxes provide auxiliary support, so that data collection, training, rollout, and evaluation share an execution distribution close to real deployment. We construct multi-source training data spanning high-frequency head tasks, high-generalization data for long-tail intents, and capability-enhancement data for reflection and memory, and introduce an error-driven data flywheel that turns failure trajectories into corrected actions, reflective explanations, and recovery demonstrations. The model is trained through a progressive three-stage pipeline of supervised fine-tuning, step-level reinforcement learning, and agentic reinforcement learning. Evaluated on public benchmarks and our in-house RealMobile, Xiaomi-GUI-0 achieves 72.0% success on RealMobile and 78.9% on AndroidWorld, while substantially improving execution stability and abnormal-state recognition in real-world tasks.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 45 of its 57 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
45 of 57 references.
Closest ancestors · 27
Same direction, newest first
- 2026Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation26 May
- 2026MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research25 May
- 2026KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation9 Apr
- 2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents15 Feb
- 2026UI-Venus-1.5 Technical Report9 Feb
- 2026MobileBench-OL: A Comprehensive Chinese Benchmark for Evaluating Mobile GUI Agents in Real-World Environment28 Jan
- 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agents26 Dec
- 2025Step-GUI Technical Report17 Dec
- 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
- 2025Mobile-Agent-v3: Fundamental Agents for GUI Automation21 Aug
- 2025UI-Venus Technical Report: Building High-performance UI Agents with RFT14 Aug
- 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
15 more
- 2025MobileGUI-RL: Advancing Mobile GUI Agent through Reinforcement Learning in Online Environment8 Jul
- 2025BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking Mechanism27 May
- 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis19 May
- 2025GUI-Shift: Enhancing VLM-Based GUI Agents through Self-supervised Reinforcement Learning18 May
- 2025MobileIPL: Enhancing Mobile Agents Thinking Process via Iterative Preference Learning18 May
- 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents14 Apr
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
- 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning27 Mar
- 2025PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PC20 Feb
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
- 2024AutoGLM: Autonomous Foundation Agents for GUIs28 Oct
- 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning14 Jun
- 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration3 Jun
- 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception29 Jan
Other references · 18
- 2024GPT-4o System Cardcited by 4,588
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
- 2025Group Sequence Policy Optimizationcited by 835
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
6 more
- 2026OpenClaw-RL: Train Any Agent Simply by Talkingcited by 90
- 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 50
- 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agentscited by 23
- 2026From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agentscited by 15
- 2025STEP: Success-Rate-Aware Trajectory-Efficient Policy Optimizationcited by 12
- 2025GUI-PRA: Process Reward Agent for GUI Taskscited by 9
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agentscited by 11
- 2026Scaling GUI Agents with Visual State Transitionscited by 4
- 2026UI-Venus-2 Technical Reportcited by 2
- 2026BlueLM-GUI Technical Report: A Real-Device-Centric Flywheel for Self-Improving Mobile GUI Agentscited by 0
- 2026From Tapping to Hopping: Augmenting Mobile GUI Agents with App-Native Deeplinkscited by 0
- 2026From Tapping to Hopping: Augmenting Mobile GUI Agents with App-Native Deeplinks25 Sep 2026
- 2026BlueLM-GUI Technical Report: A Real-Device-Centric Flywheel for Self-Improving Mobile GUI Agents11 Sep 2026
- 2026UI-Venus-2 Technical Report27 Aug 2026
- 2026Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents30 Jul 2026
- 2026Scaling GUI Agents with Visual State Transitions27 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026GUI-CEval: A Hierarchical and Comprehensive Chinese Benchmark for Mobile GUI Agents
- 2025Mobile GUI Agents under Real-world Threats: Are We There Yet?
- 2026APPSim-Bench: Bridging Real-world Apps and Reproducible Evaluation for Mobile GUI Agents
- 2025TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agents
- 2026ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents
- 2026MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Management
- 2025GUI-Robust: A Comprehensive Dataset for Testing GUI Agent Robustness in Real-World Anomalies
- 2025MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuning