Data to 14 Oct 2026

UI-Venus Technical Report: Building High-performance UI Agents with RFT

arXiv 2508.10833 ↗First posted 14 Aug 2025cs.CVcited by 93

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

We present UI-Venus, a native UI agent that takes only screenshots as input based on a multimodal large language model. UI-Venus achieves SOTA performance on both UI grounding and navigation tasks using only several hundred thousand high-quality training samples through reinforcement finetune (RFT) based on Qwen2.5-VL. Specifically, the 7B and 72B variants of UI-Venus obtain 94.1% / 50.8% and 95.3% / 61.9% on the standard grounding benchmarks, i.e., Screenspot-V2 / Pro, surpassing the previous SOTA baselines including open-source GTA1 and closed-source UI-TARS-1.5. To show UI-Venus's summary and planing ability, we also evaluate it on the AndroidWorld, an online UI navigation arena, on which our 7B and 72B variants achieve 49.1% and 65.9% success rate, also beating existing models. To achieve this, we introduce carefully designed reward functions for both UI grounding and navigation tasks and corresponding efficient data cleaning strategies. To further boost navigation performance, we propose Self-Evolving Trajectory History Alignment & Sparse Action Enhancement that refine historical reasoning traces and balances the distribution of sparse but critical actions, leading to more coherent planning and better generalization in complex UI tasks. Our contributions include the publish of SOTA open-source UI agents, comprehensive data cleaning protocols and a novel self-evolving framework for improving navigation performance, which encourage further research and development in the community. Code is available at https://github.com/inclusionAI/UI-Venus.

Key facts. Cited by 93 papers in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 50 of its 64 references that are in the map. First posted on arXiv 14 Aug 2025.

Builds on

50 of 64 references.

Closest ancestors · 26

Same direction, newest first

14 more

Other references · 24

2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 526
2025Seed1.5-VL Technical Reportcited by 373
2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 319
2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 194
2024Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigationcited by 126
2024Cradle: Empowering Foundation Agents Towards General Computer Controlcited by 117

12 more

2024A Survey on Self-Evolution of Large Language Modelscited by 91
2022A data-driven approach for learning to control computerscited by 88
2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interactioncited by 77
2025Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error Diagnosis in GUI Automationcited by 27
2025Think Twice, Click Once: Enhancing GUI Grounding via Fast and Slow Systemscited by 23
2025GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection Behaviorcited by 23
2025Towards Efficient Online Tuning of VLM Agents via Counterfactual Soft Reinforcement Learningcited by 18
2024Generalist Virtual Agents: A Survey on Autonomous Agents Across Digital Platformscited by 13
2025Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignmentcited by 12
2024E-ANT: A Large-Scale Dataset for Efficient Automatic GUI NavigaTioncited by 11
2025NaviMaster: Learning a Unified Policy for GUI and Embodied Navigation Taskscited by 10
2023Mobile User Interface Element Detection Via Adaptively Prompt Tuningcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 72.

93 papers.

Sort:

By area:

2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 91
2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agentscited by 79
2025MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environmentscited by 72
2026UI-Venus-1.5 Technical Reportcited by 41
2026AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machinescited by 21
2026KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluationcited by 21
2026GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RLcited by 16
2025VeriOS: Query-Driven Proactive Human-Agent-GUI Interaction for Trustworthy OS Agentscited by 15
2025Scaling Agents for Computer Usecited by 14
2026ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agentscited by 14
2026OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Executioncited by 13

38 more

2026LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Modelcited by 13
2026UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agentscited by 12
2025ProRe: A Proactive Reward System for GUI Agents via Reasoner-Actor Collaborationcited by 11
2025AgentProg: Empowering Long-Horizon GUI Agents with Program-Guided Context Managementcited by 11
2026UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experiencecited by 11
2025GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agentscited by 10
2025textscGUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Groundingcited by 10
2026AndroTMem: From Interaction Trajectories to Anchored Memory in Long-Horizon GUI Agentscited by 10
2026OS-Themis: A Scalable Critic Framework for Generalist GUI Rewardscited by 10
2026UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Groundingcited by 8
2025ColorAgent: Building A Robust, Personalized, and Interactive OS Agentcited by 7
2025Chain-of-Ground: Improving GUI Grounding via Iterative Reasoning and Reference Feedbackcited by 7
2025VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Taskscited by 7
2026Unified Generation and Self-Verification for Vision-Language Models via Advantage Decoupled Preference Optimizationcited by 7
2026Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolutioncited by 7
2026PIRA-Bench: A Transition from Reactive GUI Agents to GUI-based Proactive Intent Recommendation Agentscited by 7
2026Video2GUI: Synthesizing Large-Scale Interaction Trajectories for Generalized GUI Agent Pretrainingcited by 7
2025OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic Modelscited by 6
2026ProactiveMobile: A Comprehensive Benchmark for Boosting Proactive Intelligence on Mobile Devicescited by 6
2025Recon-Act: A Self-Evolving Multi-Agent Browser-Use System via Web Reconnaissance, Tool Generation, and Task Executioncited by 5
2025Hi-Agent: Hierarchical Vision-Language Agents for Mobile Device Controlcited by 5
2026Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elementscited by 5
2026CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agentscited by 5
2026VenusBench-Mobile: A Challenging and User-Centric Benchmark for Mobile GUI Agents with Capability Diagnosticscited by 5
2026Xiaomi-GUI-0 Technical Reportcited by 5
2026What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving Statescited by 5
2025Say One Thing, Do Another? Diagnosing Reasoning-Execution Gaps in VLM-Powered Mobile-Use Agentscited by 4
2026POINTS-GUI-G: GUI-Grounding Journeycited by 4
2026WebTestPilot: Agentic End-to-End Web Testing against Natural Language Specification by Inferring Oracles with Symbolized GUI Elementscited by 4
2026SecAgent: Efficient Mobile GUI Agent with Semantic Contextcited by 4
2026UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimizationcited by 4
2026What Happens Before Decoding? Prefill Determines GUI Grounding in VLMscited by 4
2026SE-GA: Memory-Augmented Self-Evolution for GUI Agentscited by 4
2026PhoneBuddy: Training Open Models for Agentic Phone Usecited by 4
2026Scaling GUI Agents with Visual State Transitionscited by 4
2026AutoFocus: Uncertainty-Aware Active Visual Search for GUI Groundingcited by 3
2026VISTA: View-Consistent Self-Verified Training for GUI Groundingcited by 3
2026MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Managementcited by 3

50 most cited of 93 All 93 (JSON)

Similar, not cited

Close in wording, no citation either way