mlsanityData to 7 Oct 2026

textscGUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Grounding

arXiv 2510.04039 ↗First posted 5 Oct 2025cs.CV cs.AIcited by 8

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Multimodal large language models (MLLMs) have markedly expanded the competence of graphical user-interface (GUI) systems, propelling them beyond controlled simulations into complex, real-world environments across diverse platforms. However, practical usefulness is still bounded by the reliability of visual grounding, i.e., mapping textual references to exact on-screen elements. This limitation prevents the system from accurately performing pointer-level actions such as clicking or dragging. To address it, we introduce GUI-Spotlight -- a model trained for image-grounded reasoning that dynamically invokes multiple specialized tools to iteratively narrow its focus to the relevant region of the screen, thereby substantially improving visual grounding accuracy. On the ScreenSpot-Pro benchmark, GUI-Spotlight trained with only 18.5K training samples achieves 52.8% accuracy, surpassing V2P-7B (50.6% with 9.6M training samples) and GTA-1-7B (50.1% with 1.56M training samples).

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 37 references that are in the map. First posted on arXiv 5 Oct 2025.

Builds on

27 of 37 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025GuirlVG: Incentivize GUI Visual Grounding via Empirical Exploration on Reinforcement Learning6 Aug
  2. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learning26 May
  3. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
  4. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  5. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
  6. 2024Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs8 Apr

Other references · 21

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  4. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  5. 2025Group Sequence Policy Optimizationcited by 788
  6. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  7. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  8. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  9. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 281
  10. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
  11. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 132
  12. 2025GTA1: GUI Test-time Scaling Agentcited by 102
9 more
  1. 2025UI-Venus Technical Report: Building High-performance UI Agents with RFTcited by 90
  2. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 83
  3. 2025Grounded Reinforcement Learning for Visual Reasoningcited by 80
  4. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 79
  5. 2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interactioncited by 68
  6. 2024VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Taskscited by 43
  7. 2025Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agentcited by 10
  8. 2024Visual Grounding with Attention-Driven Constraint Balancingcited by 7
  9. 2025InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer Interactioncited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoning
  2. 2026AdaZoom-GUI: Adaptive Zoom-based GUI Grounding with Instruction Refinement
  3. 2025MEGA-GUI: Multi-stage Enhanced Grounding Agents for GUI Elements
  4. 2025GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding
  5. 2026FocusUI: Efficient UI Grounding via Position-Preserving Visual Token Selection
  6. 2025Zoom in, Click out: Unlocking and Evaluating the Potential of Zooming for GUI Grounding
  7. 2026GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs
  8. 2026Hallucination-Free GUI Grounding via Regression-Free Layout-Aware Matching