mlsanityData to 14 Oct 2026

Xiaomi-GUI-0 Technical Report

arXiv 2606.31410 ↗First posted 30 Jun 2026cs.AIcited by 5

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

Graphical user interface (GUI) agents build on vision-language models to complete user tasks end-to-end in real applications through interface actions such as tapping, swiping, text entry, and navigation. However, existing GUI agents are trained and evaluated largely on offline trajectories, simulated environments, and standardized benchmarks. These differ substantially from real applications in interface layout, interaction logic, and abnormal-state distribution, and cannot faithfully characterize execution stability in real-world use, where account states, permission dialogs, payment authentication, and risk control continually reshape the state distribution and open a persistent gap between benchmark scores and real usability. To close this gap, we propose Xiaomi-GUI-0, a native multimodal GUI agent for real mobile environments, trained and evaluated within a real-device closed loop. At its core is a real-device-dominant hybrid infrastructure, where physical devices are the primary execution environment and sandboxes provide auxiliary support, so that data collection, training, rollout, and evaluation share an execution distribution close to real deployment. We construct multi-source training data spanning high-frequency head tasks, high-generalization data for long-tail intents, and capability-enhancement data for reflection and memory, and introduce an error-driven data flywheel that turns failure trajectories into corrected actions, reflective explanations, and recovery demonstrations. The model is trained through a progressive three-stage pipeline of supervised fine-tuning, step-level reinforcement learning, and agentic reinforcement learning. Evaluated on public benchmarks and our in-house RealMobile, Xiaomi-GUI-0 achieves 72.0% success on RealMobile and 78.9% on AndroidWorld, while substantially improving execution stability and abnormal-state recognition in real-world tasks.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 45 of its 57 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

45 of 57 references.

Closest ancestors · 27

Same direction, newest first

  1. 2026Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation26 May
  2. 2026MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research25 May
  3. 2026KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation9 Apr
  4. 2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents15 Feb
  5. 2026UI-Venus-1.5 Technical Report9 Feb
  6. 2026MobileBench-OL: A Comprehensive Chinese Benchmark for Evaluating Mobile GUI Agents in Real-World Environment28 Jan
  7. 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agents26 Dec
  8. 2025Step-GUI Technical Report17 Dec
  9. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
  10. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automation21 Aug
  11. 2025UI-Venus Technical Report: Building High-performance UI Agents with RFT14 Aug
  12. 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
15 more
  1. 2025MobileGUI-RL: Advancing Mobile GUI Agent through Reinforcement Learning in Online Environment8 Jul
  2. 2025BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking Mechanism27 May
  3. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis19 May
  4. 2025GUI-Shift: Enhancing VLM-Based GUI Agents through Self-supervised Reinforcement Learning18 May
  5. 2025MobileIPL: Enhancing Mobile Agents Thinking Process via Iterative Preference Learning18 May
  6. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents14 Apr
  7. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
  8. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning27 Mar
  9. 2025PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PC20 Feb
  10. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
  11. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  12. 2024AutoGLM: Autonomous Foundation Agents for GUIs28 Oct
  13. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning14 Jun
  14. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration3 Jun
  15. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception29 Jan

Other references · 18

  1. 2024GPT-4o System Cardcited by 4,588
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  6. 2025Qwen3-VL Technical Reportcited by 2,793
  7. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  9. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  10. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  11. 2025Group Sequence Policy Optimizationcited by 835
  12. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
6 more
  1. 2026OpenClaw-RL: Train Any Agent Simply by Talkingcited by 90
  2. 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 50
  3. 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agentscited by 23
  4. 2026From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agentscited by 15
  5. 2025STEP: Success-Rate-Aware Trajectory-Efficient Policy Optimizationcited by 12
  6. 2025GUI-PRA: Process Reward Agent for GUI Taskscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026GUI-CEval: A Hierarchical and Comprehensive Chinese Benchmark for Mobile GUI Agents
  2. 2025Mobile GUI Agents under Real-world Threats: Are We There Yet?
  3. 2026APPSim-Bench: Bridging Real-world Apps and Reproducible Evaluation for Mobile GUI Agents
  4. 2025TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agents
  5. 2026ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents
  6. 2026MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Management
  7. 2025GUI-Robust: A Comprehensive Dataset for Testing GUI Agent Robustness in Real-World Anomalies
  8. 2025MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuning