mlsanityData to 14 Oct 2026

Learning Active Perception via Self-Evolving Preference Optimization for GUI Grounding

arXiv 2509.04243 ↗First posted 4 Sep 2025cs.CV cs.AIcited by 3

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 2 more
Abstract (arXiv)

Vision Language Models (VLMs) have recently achieved significant progress in bridging visual perception and linguistic reasoning. Recently, OpenAI o3 model introduced a zoom-in search strategy that effectively elicits active perception capabilities in VLMs, improving downstream task performance. However, enabling VLMs to reason effectively over appropriate image regions remains a core challenge in GUI grounding, particularly under high-resolution inputs and complex multi-element visual interactions. In this work, we propose LASER, a self-evolving framework that progressively endows VLMs with multi-step perception capabilities, enabling precise coordinate prediction. Specifically, our approach integrate Monte Carlo quality estimation with Intersection-over-Union (IoU)-based region quality evaluation to jointly encourage both accuracy and diversity in constructing high-quality preference data. This combination explicitly guides the model to focus on instruction-relevant key regions while adaptively allocating reasoning steps based on task complexity. Comprehensive experiments on the ScreenSpot Pro and ScreenSpot-v2 benchmarks demonstrate consistent performance gains, validating the effectiveness of our method. Furthermore, when fine-tuned on GTA1-7B, LASER achieves a score of 55.7 on the ScreenSpot-Pro benchmark, establishing a new state-of-the-art (SoTA) among 7B-scale models.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 28 references that are in the map. First posted on arXiv 4 Sep 2025.

Builds on

22 of 28 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs21 May
  2. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents14 Apr
  3. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
  4. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  5. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
  6. 2023CogAgent: A Visual Language Model for GUI Agents14 Dec

Other references · 16

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  4. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  5. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  6. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  7. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  8. 2024Understanding the planning of LLM agents: A surveycited by 421
  9. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
  10. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 278
  11. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  12. 2024Aria-UI: Visual Grounding for GUI Instructionscited by 140
4 more
  1. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 139
  2. 2025GTA1: GUI Test-time Scaling Agentcited by 106
  3. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 81
  4. 2025ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learningcited by 43

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026GUI-Eyes: Tool-Augmented Perception for Visual Grounding in GUI Agents
  2. 2026Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding
  3. 2026AutoFocus: Uncertainty-Aware Active Visual Search for GUI Grounding
  4. 2025R-VLM: Region-Aware Vision Language Model for Precise GUI Grounding
  5. 2025MEGA-GUI: Multi-stage Enhanced Grounding Agents for GUI Elements
  6. 2025DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoning
  7. 2025How Auxiliary Reasoning Unleashes GUI Grounding in VLMs
  8. 2026Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets