Zoom in, Click out: Unlocking and Evaluating the Potential of Zooming for GUI Grounding
Not on the map. Why?
Abstract (arXiv)
Grounding is a fundamental capability for building graphical user interface (GUI) agents. Although existing approaches rely on large-scale bounding box supervision, they still face various challenges, such as cross-platform generalization, complex layout analysis, and fine-grained element localization. In this paper, we investigate zoom as a strong yet underexplored prior for GUI grounding, and propose a training-free method, ZoomClick. By characterizing four key properties of zoom (i.e., pre-zoom, depth, shrink size, minimal crop size), we unlock its full capabilities for dynamic spatial focusing and adaptive context switching. Experiments demonstrate that our method significantly boosts the performance of both general vision-language and specialized GUI grounding models, achieving state-of-the-art results on several mainstream benchmarks; for example, UI-Venus-72B attains a 73.1% success rate on ScreenSpot-Pro. Furthermore, we present GUIZoom-Bench, a benchmark for evaluating model adaptability to zoom, aiming to inspire future research on improving zoom for further training and test-time scaling in GUI grounding tasks.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 37 of its 50 references that are in the map. First posted on arXiv 5 Dec 2025.
Builds on
37 of 50 references.
References · 37
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 370
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
25 more
- 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
- 2024OmniParser for Pure Vision Based GUI Agentcited by 193
- 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
- 2024Aria-UI: Visual Grounding for GUI Instructionscited by 136
- 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 132
- 2025Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolutioncited by 114
- 2023LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Modelscited by 113
- 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agentscited by 89
- 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 73
- 2024CogCoM: A Visual Language Model with Chain-of-Manipulations Reasoningcited by 64
- 2025DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoningcited by 29
- 2025Visual Test-time Scaling for GUI Agent Groundingcited by 27
- 2025R-VLM: Region-Aware Vision Language Model for Precise GUI Groundingcited by 20
- 2023FM-OV3D: Foundation Model-based Cross-modal Knowledge Blending for Open-Vocabulary 3D Detectioncited by 19
- 2024Improved GUI Grounding via Iterative Narrowingcited by 15
- 2024Attention-driven GUI Grounding: Leveraging Pretrained Multimodal Large Language Models without Fine-Tuningcited by 13
- 2025PAL-UI: Planning with Active Look-back for Vision-Based GUI Agentscited by 11
- 2025Efficient Test-Time Scaling for Small Vision-Language Modelscited by 11
- 2024Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspectivecited by 10
- 2025GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agentscited by 10
- 2024Embedded Visual Prompt Tuningcited by 7
- 2025GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Groundingcited by 6
- 2025Generalist Scanner Meets Specialist Locator: A Synergistic Coarse-to-Fine Framework for Robust GUI Groundingcited by 3
- 2025Test-Time-Scaling for Zero-Shot Diagnosis with Visual-Language Reasoningcited by 2
- 2025Exploring Representation Invariance in Finetuningcited by 1
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026UI-Venus-1.5 Technical Reportcited by 39
- 2026Zoom Consistency: A Free Confidence Signal in Multi-Step Visual Grounding Pipelinescited by 4
- 2026AutoFocus: Uncertainty-Aware Active Visual Search for GUI Groundingcited by 1
- 2026What Happens Before Decoding? Prefill Determines GUI Grounding in VLMscited by 1
- 2026One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Groundingcited by 1
- 2026GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMscited by 0
- 2026Probe to Act: Elevating Browser-Use Agent via Active Visual Probingcited by 0
- 2026Probe to Act: Elevating Browser-Use Agent via Active Visual Probing27 Sep 2026
- 2026GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs4 Aug 2026
- 2026One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding29 Jun 2026
- 2026What Happens Before Decoding? Prefill Determines GUI Grounding in VLMs10 May 2026
- 2026AutoFocus: Uncertainty-Aware Active Visual Search for GUI Grounding4 May 2026
- 2026Zoom Consistency: A Free Confidence Signal in Multi-Step Visual Grounding Pipelines15 Apr 2026
- 2026UI-Venus-1.5 Technical Report9 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2026AdaZoom-GUI: Adaptive Zoom-based GUI Grounding with Instruction Refinement
- 2025textscGUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Grounding
- 2025MEGA-GUI: Multi-stage Enhanced Grounding Agents for GUI Elements
- 2026Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elements
- 2026LookAgain: Closed-Loop GUI Grounding with Visually Grounded Reflection
- 2026GoClick: Lightweight Element Grounding Model for Autonomous GUI Interaction
- 2025Beyond Clicking:A Step Towards Generalist GUI Grounding via Text Dragging
- 2026FocusUI: Efficient UI Grounding via Position-Preserving Visual Token Selection