# MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis Search

arXiv: https://arxiv.org/abs/2610.17792 · HTML: https://mlsanity.com/paper/2610.17792/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Placed in

- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)

## Builds on (most cited first, 26 of 26)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [GPT-4V(ision) is a Generalist Web Agent, if Grounded](https://mlsanity.com/paper/2401.01614/) (2024)
- [Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents](https://mlsanity.com/paper/2410.05243/) (2024)
- [ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use](https://mlsanity.com/paper/2504.07981/) (2025)
- [Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis](https://mlsanity.com/paper/2505.13227/) (2025)
- [GTA1: GUI Test-time Scaling Agent](https://mlsanity.com/paper/2507.05791/) (2025)
- [VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?](https://mlsanity.com/paper/2404.05955/) (2024)
- [DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoning](https://mlsanity.com/paper/2507.00008/) (2025)
- [Visual Test-time Scaling for GUI Agent Grounding](https://mlsanity.com/paper/2505.00684/) (2025)
- [ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and Search](https://mlsanity.com/paper/2505.15259/) (2025)
- [textscGUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Grounding](https://mlsanity.com/paper/2510.04039/) (2025)
- [UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Grounding](https://mlsanity.com/paper/2604.14113/) (2026)
- [Chain-of-Ground: Improving GUI Grounding via Iterative Reasoning and Reference Feedback](https://mlsanity.com/paper/2512.01979/) (2025)
- [AdaZoom-GUI: Adaptive Zoom-based GUI Grounding with Instruction Refinement](https://mlsanity.com/paper/2603.17441/) (2026)
- [Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elements](https://mlsanity.com/paper/2603.14448/) (2026)
- [DRS-GUI: Dynamic Region Search for Training-Free GUI Grounding](https://mlsanity.com/paper/2605.15542/) (2026)
- [Generalist Scanner Meets Specialist Locator: A Synergistic Coarse-to-Fine Framework for Robust GUI Grounding](https://mlsanity.com/paper/2509.24133/) (2025)
- [What Happens Before Decoding? Prefill Determines GUI Grounding in VLMs](https://mlsanity.com/paper/2605.12549/) (2026)
- [MEGA-GUI: Multi-stage Enhanced Grounding Agents for GUI Elements](https://mlsanity.com/paper/2511.13087/) (2025)
- [AutoFocus: Uncertainty-Aware Active Visual Search for GUI Grounding](https://mlsanity.com/paper/2605.02630/) (2026)
- [BAMI: Training-Free Bias Mitigation in GUI Grounding](https://mlsanity.com/paper/2605.06664/) (2026)
- [Mitigating Coordinate Prediction Bias from Positional Encoding Failures](https://mlsanity.com/paper/2510.22102/) (2025)
- [WinDeskGround: A Benchmark for Robust GUI Grounding in Complex Multi-Window Desktop Environments](https://mlsanity.com/paper/2605.16402/) (2026)
- [One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding](https://mlsanity.com/paper/2606.30084/) (2026)
- [Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation](https://mlsanity.com/paper/2608.11191/) (2026)

## Abstract (arXiv)

GUI grounding enables web agents to translate natural-language instructions into actionable interface locations. However, dense layouts and small, visually similar controls can lead models toward plausible but incorrect elements, while refinement around a single prediction may leave alternative targets unexplored. We propose the Multi-Hypothesis Verification Framework (MHVF) for training-free GUI grounding. It combines multi-hypothesis exploration with local visual verification using a single pretrained vision-language model. To explore alternative targets, MHVF constructs spatial hypotheses from multiple full-screen predictions and examines the corresponding regions through zoomed re-localization. To resolve ambiguity among the resulting candidates, it verifies their agreement with the instruction in marked local views. Verification scores then guide a second round of local search and rank candidates retained from both search levels for final selection. Experiments with five general-purpose and GUI-specific models across eight benchmark settings demonstrate improvements in grounding accuracy in most settings. MHVF improves UI-Venus-Ground-7B by 4.3 and 2.1 percentage points on Mind2Web and VisualWebBench, respectively, and raises Qwen2.5-VL-7B's accuracy on ScreenSpot-Pro from 23.6% to 53.6%.