mlsanityData to 14 Oct 2026

Zoom Consistency: A Free Confidence Signal in Multi-Step Visual Grounding Pipelines

arXiv 2604.15376 ↗First posted 15 Apr 2026cs.CV cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Multi-step zoom-in pipelines are widely used for GUI grounding, yet the intermediate predictions they produce are typically discarded after coordinate remapping. We observe that these intermediate outputs contain a useful confidence signal for free: zoom consistency, the distance between a model's step-2 prediction and the crop center. Unlike log-probabilities or token-level uncertainty, zoom consistency is a geometric quantity in a shared coordinate space, making it directly comparable across architecturally different VLMs without calibration. We prove this quantity is a linear estimator of step-1 spatial error under idealized conditions (perfect step-2, target within crop) and show it correlates with prediction correctness across two VLMs (AUC = 0.60; Spearman rho = -0.14, p < 10^{-6} for KV-Ground-8B; rho = -0.11, p = 0.0003 for Qwen3.5-27B). The correlation is small but consistent across models, application categories, and operating systems. As a proof-of-concept, we use zoom consistency to route between a specialist and generalist model, capturing 16.5% of the oracle headroom between them (+0.8%, McNemar p = 0.19). Code is available at https://github.com/omxyz/zoom-consistency-routing.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 16 of its 19 references that are in the map. First posted on arXiv 15 Apr 2026.

Builds on

16 of 19 references.

References · 16

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  2. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  3. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  4. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 382
  5. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 378
  6. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 254
  7. 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMscited by 96
  8. 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agentscited by 95
  9. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 81
  10. 2025Visual Test-time Scaling for GUI Agent Groundingcited by 30
  11. 2025Test-Time Reinforcement Learning for GUI Grounding via Region Consistencycited by 21
  12. 2025Zoom in, Click out: Unlocking and Evaluating the Potential of Zooming for GUI Groundingcited by 8
4 more
  1. 2026Adaptive Vision-Language Model Routing for Computer Use Agentscited by 8
  2. 2026AdaZoom-GUI: Adaptive Zoom-based GUI Grounding with Instruction Refinementcited by 6
  3. 2025Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMscited by 4
  4. 2025MEGA-GUI: Multi-stage Enhanced Grounding Agents for GUI Elementscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026GUI-Perturbed: Domain Randomization Reveals Systematic Brittleness in GUI Grounding Models
  2. 2026LookAgain: Closed-Loop GUI Grounding with Visually Grounded Reflection
  3. 2026One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding
  4. 2026AutoFocus: Uncertainty-Aware Active Visual Search for GUI Grounding
  5. 2026GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs
  6. 2026AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Grounding
  7. 2025Improving GUI Grounding with Explicit Position-to-Coordinate Mapping
  8. 2026Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets