# UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Grounding

arXiv: https://arxiv.org/abs/2604.14113 · HTML: https://mlsanity.com/paper/2604.14113/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 38 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Builds on (most cited first, 30 of 33)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [CogAgent: A Visual Language Model for GUI Agents](https://mlsanity.com/paper/2312.08914/) (2023)
- [SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents](https://mlsanity.com/paper/2401.10935/) (2024)
- [Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents](https://mlsanity.com/paper/2410.05243/) (2024)
- [OS-ATLAS: A Foundation Action Model for Generalist GUI Agents](https://mlsanity.com/paper/2410.23218/) (2024)
- [Seed1.5-VL Technical Report](https://mlsanity.com/paper/2505.07062/) (2025)
- [Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction](https://mlsanity.com/paper/2412.04454/) (2024)
- [ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use](https://mlsanity.com/paper/2504.07981/) (2025)
- [ShowUI: One Vision-Language-Action Model for GUI Visual Agent](https://mlsanity.com/paper/2411.17465/) (2024)
- [Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration](https://mlsanity.com/paper/2406.01014/) (2024)
- [Agent S: An Open Agentic Framework that Uses Computers Like a Human](https://mlsanity.com/paper/2410.08164/) (2024)
- [UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning](https://mlsanity.com/paper/2503.21620/) (2025)
- [Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents](https://mlsanity.com/paper/2504.00906/) (2025)
- [Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis](https://mlsanity.com/paper/2505.13227/) (2025)
- [GTA1: GUI Test-time Scaling Agent](https://mlsanity.com/paper/2507.05791/) (2025)
- [GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents](https://mlsanity.com/paper/2506.03143/) (2025)
- [UI-Venus Technical Report: Building High-performance UI Agents with RFT](https://mlsanity.com/paper/2508.10833/) (2025)
- [Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning](https://mlsanity.com/paper/2505.12370/) (2025)
- [UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction](https://mlsanity.com/paper/2503.15661/) (2025)
- [A Survey on (M)LLM-Based GUI Agents](https://mlsanity.com/paper/2504.13865/) (2025)
- [AppAgentX: Evolving GUI Agents as Proficient Smartphone Users](https://mlsanity.com/paper/2503.02268/) (2025)
- [UI-S1: Advancing GUI Automation via Semi-online Reinforcement Learning](https://mlsanity.com/paper/2509.11543/) (2025)
- [DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoning](https://mlsanity.com/paper/2507.00008/) (2025)
- [Visual Test-time Scaling for GUI Agent Grounding](https://mlsanity.com/paper/2505.00684/) (2025)
- [InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy Optimization](https://mlsanity.com/paper/2508.05731/) (2025)
- [Think Twice, Click Once: Enhancing GUI Grounding via Fast and Slow Systems](https://mlsanity.com/paper/2503.06470/) (2025)
- [Test-Time Reinforcement Learning for GUI Grounding via Region Consistency](https://mlsanity.com/paper/2508.05615/) (2025)
- [Improved GUI Grounding via Iterative Narrowing](https://mlsanity.com/paper/2411.13591/) (2024)

## Built on by (most cited first, 8 of 8)

- [What Happens Before Decoding? Prefill Determines GUI Grounding in VLMs](https://mlsanity.com/paper/2605.12549/) (2026)
- [GUI-C2: Coarse-to-Fine GUI Grounding via Difficulty-Aware Reinforcement Learning](https://mlsanity.com/paper/2605.30884/) (2026)
- [Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets](https://mlsanity.com/paper/2606.25760/) (2026)
- [One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding](https://mlsanity.com/paper/2606.30084/) (2026)
- [Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Grounding](https://mlsanity.com/paper/2609.15307/) (2026)
- [Recurrent Latent Visual Search for GUI Grounding](https://mlsanity.com/paper/2610.05185/) (2026)
- [ConClick: Element-Wise Contrastive Learning Improves GUI Grounding](https://mlsanity.com/paper/2610.13268/) (2026)
- [MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis Search](https://mlsanity.com/paper/2610.17792/) (2026)

## Abstract (arXiv)

GUI grounding, which localizes interface elements from screenshots given natural language queries, remains challenging for small icons and dense layouts. Test-time zoom-in methods improve localization by cropping and re-running inference at higher resolution, but apply cropping uniformly across all instances with fixed crop sizes, ignoring whether the model is actually uncertain on each case. We propose UI-Zoomer, a training-free adaptive zoom-in framework that treats both the trigger and scale of zoom-in as a prediction uncertainty quantification problem. A confidence-aware gate fuses spatial consensus among stochastic candidates with token-level generation confidence to selectively trigger zoom-in only when localization is uncertain. When triggered, an uncertainty-driven crop sizing module decomposes prediction variance into inter-sample positional spread and intra-sample box extent, deriving a per-instance crop radius via the law of total variance. Extensive experiments on ScreenSpot-Pro, UI-Vision, and ScreenSpot-v2 demonstrate consistent improvements over strong baselines across multiple model architectures, achieving gains of up to +13.4%, +10.3%, and +4.2% respectively, with no additional training required.