mlsanityData to 14 Oct 2026

One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding

arXiv 2606.30084 ↗First posted 29 Jun 2026cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

MLLM-based GUI grounding methods commonly formulate target localization as autoregressive coordinate generation, enabling models to leverage the strong instruction-following and semantic understanding capabilities of MLLMs. However, this formulation requires the model to retain region-level target evidence while decoding coordinate tokens with the spatial precision demanded by GUI clicking. Our diagnostic analysis reveals that target-region awareness emerges in intermediate decoder layers but is neither retained nor translated into the final coordinate prediction. Existing ZoomIn-style methods address this issue through an external crop-and-rerun pass, which improves localization but increases end-to-end latency and computational cost. To retain the accuracy benefits of two-pass zooming without this extra cost, we propose InnerZoom, a single-forward framework for cross-layer evidence bridging. InnerZoom transforms target-related cues from the original forward pass into a compact cross-layer evidence state, then preserves, refines, and reinjects this state throughout later decoding layers to guide coordinate prediction. Extensive experimental results suggest that InnerZoom-4B achieves state-of-the-art performance on all six GUI grounding benchmarks, obtaining 64.7 on OSWorld-G, 40.2 on UI-Vision, 73.1 on OSWorld-GR, and 87.6 on MMBench-GUI, surpassing the previous best results by 4.1, 3.2, 2.9, and 2.3 points, respectively. Under a controlled 4B setting, InnerZoom improves the same SFT+RL baseline by 5.3 points on average and outperforms two-pass ZoomIn by 1.3 points on average, while reducing end-to-end latency by up to 31.8% and TFLOPs by about 29%. Code and models will be publicly available.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 63 of its 67 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

63 of 67 references.

References · 63

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2025Qwen3-VL Technical Reportcited by 2,793
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  7. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  8. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  9. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  10. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  11. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  12. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 382
38 more
  1. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 378
  2. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
  3. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 254
  4. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 232
  5. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 215
  6. 2024On the Effects of Data Scale on UI Control Agentscited by 194
  7. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 190
  8. 2025OpenCUA: Open Foundations for Computer-Use Agentscited by 160
  9. 2024OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Webcited by 157
  10. 2024Aria-UI: Visual Grounding for GUI Instructionscited by 140
  11. 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inferencecited by 139
  12. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 139
  13. 2024AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agentscited by 113
  14. 2025GTA1: GUI Test-time Scaling Agentcited by 106
  15. 2024GUI Agents: A Surveycited by 103
  16. 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agentscited by 95
  17. 2025UI-Venus Technical Report: Building High-performance UI Agents with RFTcited by 93
  18. 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 91
  19. 2025AgentNet: Decentralized Evolutionary Coordination for LLM-based Multi-Agent Systemscited by 90
  20. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 81
  21. 2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agentscited by 79
  22. 2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interactioncited by 77
  23. 2025MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agentscited by 67
  24. 2025GUI-G2: Gaussian Reward Modeling for GUI Groundingcited by 63
  25. 2026EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experiencecited by 53
  26. 2026UI-Venus-1.5 Technical Reportcited by 41
  27. 2025DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoningcited by 31
  28. 2025Visual Test-time Scaling for GUI Agent Groundingcited by 30
  29. 2025InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy Optimizationcited by 27
  30. 2025Phi-Ground Tech Report: Advancing Perception in GUI Groundingcited by 25
  31. 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoningcited by 22
  32. 2025Test-Time Reinforcement Learning for GUI Grounding via Region Consistencycited by 21
  33. 2025Ferret-UI Lite: Lessons from Building Small On-Device GUI Agentscited by 16
  34. 2025UGround: Towards Unified Visual Grounding with Unrolled Transformerscited by 14
  35. 2025ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and Searchcited by 13
  36. 2025MVP: Multiple View Prediction Improves GUI Groundingcited by 12
  37. 2025Towards Understanding Visual Grounding in Visual Language Modelscited by 10
  38. 2025textscGUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Groundingcited by 10

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Seg2Click: Learning Pixel-Level Action Masks for GUI Visual Grounding
  2. 2026LookAgain: Closed-Loop GUI Grounding with Visually Grounded Reflection
  3. 2026GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs
  4. 2025Generalist Scanner Meets Specialist Locator: A Synergistic Coarse-to-Fine Framework for Robust GUI Grounding
  5. 2026AutoFocus: Uncertainty-Aware Active Visual Search for GUI Grounding
  6. 2026GoClick: Lightweight Element Grounding Model for Autonomous GUI Interaction
  7. 2026Hallucination-Free GUI Grounding via Regression-Free Layout-Aware Matching
  8. 2026Trifuse: Enhancing Attention-Based GUI Grounding via Multimodal Fusion