VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Tasks
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
GUI grounding is a critical component in building capable GUI agents. However, existing grounding benchmarks suffer from significant limitations: they either provide insufficient data volume and narrow domain coverage, or focus excessively on a single platform and require highly specialized domain knowledge. In this work, we present VenusBench-GD, a comprehensive, bilingual benchmark for GUI grounding that spans multiple platforms, enabling hierarchical evaluation for real-word applications. VenusBench-GD contributes as follows: (i) we introduce a large-scale, cross-platform benchmark with extensive coverage of applications, diverse UI elements, and rich annotated data, (ii) we establish a high-quality data construction pipeline for grounding tasks, achieving higher annotation accuracy than existing benchmarks, and (iii) we extend the scope of element grounding by proposing a hierarchical task taxonomy that divides grounding into basic and advanced categories, encompassing six distinct subtasks designed to evaluate models from complementary perspectives. Our experimental findings reveal critical insights: general-purpose multimodal models now match or even surpass specialized GUI models on basic grounding tasks. In contrast, advanced tasks, still favor GUI-specialized models, though they exhibit significant overfitting and poor robustness. These results underscore the necessity of comprehensive, multi-tiered evaluation frameworks.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 33 of its 39 references that are in the map. First posted on arXiv 18 Dec 2025.
Builds on
33 of 39 references.
Closest ancestors · 17
Same direction, newest first
- 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoning23 Oct
- 2025UI-Venus Technical Report: Building High-performance UI Agents with RFT14 Aug
- 2025MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents25 Jul
- 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents3 Jun
- 2025AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning2 Jun
- 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis19 May
- 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning18 May
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
- 2025Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks20 Jan
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
- 2024Agent S: An Open Agentic Framework that Uses Computers Like a Human10 Oct
5 more
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr
- 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception29 Jan
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents17 Jan
- 2023CogAgent: A Visual Language Model for GUI Agents14 Dec
Other references · 16
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
- 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 186
- 2024WebCanvas: Benchmarking Web Agents in Online Environmentscited by 121
- 2025GTA1: GUI Test-time Scaling Agentcited by 106
- 2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interactioncited by 77
- 2025GUI-G2: Gaussian Reward Modeling for GUI Groundingcited by 63
4 more
- 2024E-ANT: A Large-Scale Dataset for Efficient Automatic GUI NavigaTioncited by 11
- 2023Mobile User Interface Element Detection Via Adaptively Prompt Tuningcited by 8
- 2025V2P: Visual Attention Calibration for GUI Grounding via Background Suppression and Center Peakingcited by 6
- 2025Recon-Act: A Self-Evolving Multi-Agent Browser-Use System via Web Reconnaissance, Tool Generation, and Task Executioncited by 5
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026UI-Venus-1.5 Technical Reportcited by 41
- 2026CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agentscited by 5
- 2026MedSPOT: A Workflow-Aware Sequential Grounding Benchmark for Clinical GUIcited by 3
- 2026MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillationcited by 3
- 2026UI-Venus-2 Technical Reportcited by 2
- 2026LookAgain: Closed-Loop GUI Grounding with Visually Grounded Reflectioncited by 0
- 2026Do GUI Agents Know When Not to Act? Enabling Conflict-Aware Termination for Multimodal GUI Agentscited by 0
- 2026Do GUI Agents Know When Not to Act? Enabling Conflict-Aware Termination for Multimodal GUI Agents3 Sep 2026
- 2026UI-Venus-2 Technical Report27 Aug 2026
- 2026LookAgain: Closed-Loop GUI Grounding with Visually Grounded Reflection10 Aug 2026
- 2026MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillation31 Jul 2026
- 2026CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents25 Mar 2026
- 2026MedSPOT: A Workflow-Aware Sequential Grounding Benchmark for Clinical GUI20 Mar 2026
- 2026UI-Venus-1.5 Technical Report9 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2026WinDeskGround: A Benchmark for Robust GUI Grounding in Complex Multi-Window Desktop Environments
- 2025Chain-of-Ground: Improving GUI Grounding via Iterative Reasoning and Reference Feedback
- 2025MEGA-GUI: Multi-stage Enhanced Grounding Agents for GUI Elements
- 2025Generalist Scanner Meets Specialist Locator: A Synergistic Coarse-to-Fine Framework for Robust GUI Grounding
- 2026AdaZoom-GUI: Adaptive Zoom-based GUI Grounding with Instruction Refinement
- 2024Aria-UI: Visual Grounding for GUI Instructions
- 2026MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis Search
- 2025DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoning