MVP: Multiple View Prediction Improves GUI Grounding
Not on the map. Why?
Abstract (arXiv)
GUI grounding, which translates natural language instructions into precise pixel coordinates, is essential for developing practical GUI agents. However, we observe that existing grounding models exhibit significant coordinate prediction instability, minor visual perturbations (e.g. cropping a few pixels) can drastically alter predictions, flipping results between correct and incorrect. This instability severely undermines model performance, especially for samples with high-resolution and small UI elements. To address this issue, we propose Multi-View Prediction (MVP), a training-free framework that enhances grounding performance through multi-view inference. Our key insight is that while single-view predictions may be unstable, aggregating predictions from multiple carefully cropped views can effectively distinguish correct coordinates from outliers. MVP comprises two components: (1) Attention-Guided View Proposal, which derives diverse views guided by instruction-to-image attention scores, and (2) Multi-Coordinates Clustering, which ensembles predictions by selecting the centroid of the densest spatial cluster. Extensive experiments demonstrate MVP's effectiveness across various models and benchmarks. Notably, on ScreenSpot-Pro, MVP boosts UI-TARS-1.5-7B to 56.1%, GTA1-7B to 61.7%, Qwen3VL-8B-Instruct to 65.3%, and Qwen3VL-32B-Instruct to 74.0%. The code is available at https://github.com/ZJUSCL/MVP.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 31 of its 47 references that are in the map. First posted on arXiv 9 Dec 2025.
Builds on
31 of 47 references.
References · 31
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
- 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 370
- 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 283
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
- 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
- 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
- 2024Aria-UI: Visual Grounding for GUI Instructionscited by 136
19 more
- 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 132
- 2024GUI Agents with Foundation Models: A Comprehensive Surveycited by 110
- 2024GUI Agents: A Surveycited by 101
- 2024VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?cited by 98
- 2024DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Modelscited by 93
- 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agentscited by 89
- 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 79
- 2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interactioncited by 68
- 2024PruneVid: Visual Token Pruning for Efficient Video Large Language Modelscited by 64
- 2025A Survey on (M)LLM-Based GUI Agentscited by 61
- 2025Visual Test-time Scaling for GUI Agent Groundingcited by 27
- 2025Phi-Ground Tech Report: Advancing Perception in GUI Groundingcited by 24
- 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoningcited by 20
- 2025FlexSelect: Flexible Token Selection for Efficient Long Video Understandingcited by 17
- 2024Attention-driven GUI Grounding: Leveraging Pretrained Multimodal Large Language Models without Fine-Tuningcited by 13
- 2025Learning GUI Grounding with Spatial Reasoning from Visual Feedbackcited by 12
- 2025GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agentscited by 10
- 2025Generalist Scanner Meets Specialist Locator: A Synergistic Coarse-to-Fine Framework for Robust GUI Groundingcited by 3
- 2025Improving GUI Grounding with Explicit Position-to-Coordinate Mappingcited by 3
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026UI-Venus-1.5 Technical Reportcited by 39
- 2026VISTA: View-Consistent Self-Verified Training for GUI Groundingcited by 2
- 2026What Happens Before Decoding? Prefill Determines GUI Grounding in VLMscited by 1
- 2026One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Groundingcited by 1
- 2026UI-Venus-2 Technical Reportcited by 1
- 2026From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use Agentscited by 1
- 2026GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMscited by 0
- 2026LookAgain: Closed-Loop GUI Grounding with Visually Grounded Reflectioncited by 0
- 2026RankGround: Efficient High-Resolution GUI Grounding via Lightweight Reranker-Guided Crop Selectioncited by 0
- 2026ScreenHaystack: Finding Blind Zones in GUI Groundingcited by 0
- 2026ScreenHaystack: Finding Blind Zones in GUI Grounding25 Sep 2026
- 2026RankGround: Efficient High-Resolution GUI Grounding via Lightweight Reranker-Guided Crop Selection16 Sep 2026
- 2026From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use Agents4 Sep 2026
- 2026UI-Venus-2 Technical Report27 Aug 2026
- 2026LookAgain: Closed-Loop GUI Grounding with Visually Grounded Reflection10 Aug 2026
- 2026GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs4 Aug 2026
- 2026One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding29 Jun 2026
- 2026VISTA: View-Consistent Self-Verified Training for GUI Grounding12 Jun 2026
- 2026What Happens Before Decoding? Prefill Determines GUI Grounding in VLMs10 May 2026
- 2026UI-Venus-1.5 Technical Report9 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2025MEGA-GUI: Multi-stage Enhanced Grounding Agents for GUI Elements
- 2025DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoning
- 2026Hallucination-Free GUI Grounding via Regression-Free Layout-Aware Matching
- 2025ZonUI-3B: A Lightweight Vision-Language Model for Cross-Resolution GUI Grounding
- 2025Chain-of-Ground: Improving GUI Grounding via Iterative Reasoning and Reference Feedback
- 2026AdaZoom-GUI: Adaptive Zoom-based GUI Grounding with Instruction Refinement
- 2025textscGUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Grounding
- 2026FocusUI: Efficient UI Grounding via Position-Preserving Visual Token Selection