# Zoom Consistency: A Free Confidence Signal in Multi-Step Visual Grounding Pipelines

arXiv: https://arxiv.org/abs/2604.15376 · HTML: https://mlsanity.com/paper/2604.15376/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 19 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Builds on (most cited first, 16 of 16)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents](https://mlsanity.com/paper/2401.10935/) (2024)
- [Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents](https://mlsanity.com/paper/2410.05243/) (2024)
- [OS-ATLAS: A Foundation Action Model for Generalist GUI Agents](https://mlsanity.com/paper/2410.23218/) (2024)
- [ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use](https://mlsanity.com/paper/2504.07981/) (2025)
- [Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs](https://mlsanity.com/paper/2505.15436/) (2025)
- [GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents](https://mlsanity.com/paper/2506.03143/) (2025)
- [Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning](https://mlsanity.com/paper/2505.12370/) (2025)
- [Visual Test-time Scaling for GUI Agent Grounding](https://mlsanity.com/paper/2505.00684/) (2025)
- [Test-Time Reinforcement Learning for GUI Grounding via Region Consistency](https://mlsanity.com/paper/2508.05615/) (2025)
- [Zoom in, Click out: Unlocking and Evaluating the Potential of Zooming for GUI Grounding](https://mlsanity.com/paper/2512.05941/) (2025)
- [Adaptive Vision-Language Model Routing for Computer Use Agents](https://mlsanity.com/paper/2603.12823/) (2026)
- [AdaZoom-GUI: Adaptive Zoom-based GUI Grounding with Instruction Refinement](https://mlsanity.com/paper/2603.17441/) (2026)
- [Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs](https://mlsanity.com/paper/2510.00705/) (2025)
- [MEGA-GUI: Multi-stage Enhanced Grounding Agents for GUI Elements](https://mlsanity.com/paper/2511.13087/) (2025)

## Built on by (most cited first, 4 of 4)

- [TACO: Tool-Augmented Credit Optimization for Agentic Tool Use](https://mlsanity.com/paper/2606.30251/) (2026)
- [It's the Decoding Format, Not the Perturbation: Auditing Consistency-Based Selection for Vision-Language Test-Time Scaling](https://mlsanity.com/paper/2608.01207/) (2026)
- [When Does Consensus Mean Correctness? Measuring the Agreement-Accuracy Coupling with Semantics-Preserving Re-Rendering](https://mlsanity.com/paper/2608.05670/) (2026)
- [Consistency Has a Computable Blind Spot: A Commutation Theory of Label-Free Reliability for Vision-Language Figure Reading](https://mlsanity.com/paper/2608.05675/) (2026)

## Abstract (arXiv)

Multi-step zoom-in pipelines are widely used for GUI grounding, yet the intermediate predictions they produce are typically discarded after coordinate remapping. We observe that these intermediate outputs contain a useful confidence signal for free: zoom consistency, the distance between a model's step-2 prediction and the crop center. Unlike log-probabilities or token-level uncertainty, zoom consistency is a geometric quantity in a shared coordinate space, making it directly comparable across architecturally different VLMs without calibration. We prove this quantity is a linear estimator of step-1 spatial error under idealized conditions (perfect step-2, target within crop) and show it correlates with prediction correctness across two VLMs (AUC = 0.60; Spearman rho = -0.14, p < 10^{-6} for KV-Ground-8B; rho = -0.11, p = 0.0003 for Qwen3.5-27B). The correlation is small but consistent across models, application categories, and operating systems. As a proof-of-concept, we use zoom consistency to route between a specialist and generalist model, capturing 16.5% of the oracle headroom between them (+0.8%, McNemar p = 0.19). Code is available at https://github.com/omxyz/zoom-consistency-routing.