mlsanityData to 7 Oct 2026

POINTS-GUI-G: GUI-Grounding Journey

arXiv 2602.06391 ↗First posted 6 Feb 2026cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

The rapid advancement of vision-language models has catalyzed the emergence of GUI agents, which hold immense potential for automating complex tasks, from online shopping to flight booking, thereby alleviating the burden of repetitive digital workflows. As a foundational capability, GUI grounding is typically established as a prerequisite for end-to-end task execution. It enables models to precisely locate interface elements, such as text and icons, to perform accurate operations like clicking and typing. Unlike prior works that fine-tune models already possessing strong spatial awareness (e.g., Qwen3-VL), we aim to master the full technical pipeline by starting from a base model with minimal grounding ability, such as POINTS-1.5. We introduce POINTS-GUI-G-8B, which achieves state-of-the-art performance with scores of 59.9 on ScreenSpot-Pro, 66.0 on OSWorld-G, 95.7 on ScreenSpot-v2, and 49.9 on UI-Vision. Our model's success is driven by three key factors: (1) Refined Data Engineering, involving the unification of diverse open-source datasets format alongside sophisticated strategies for augmentation, filtering, and difficulty grading; (2) Improved Training Strategies, including continuous fine-tuning of the vision encoder to enhance perceptual accuracy and maintaining resolution consistency between training and inference; and (3) Reinforcement Learning (RL) with Verifiable Rewards. While RL is traditionally used to bolster reasoning, we demonstrate that it significantly improves precision in the perception-intensive GUI grounding task. Furthermore, GUI grounding provides a natural advantage for RL, as rewards are easily verifiable and highly accurate.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 62 of its 69 references that are in the map. First posted on arXiv 6 Feb 2026.

Builds on

62 of 69 references.

References · 62

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2025Qwen2.5-VL Technical Reportcited by 5,418
  9. 2024Qwen2.5 Technical Reportcited by 4,713
  10. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  11. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  12. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
38 more
  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2025Qwen3-VL Technical Reportcited by 2,534
  3. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  4. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  5. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  7. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  8. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  9. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  10. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  11. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 446
  12. 2025Kimi-VL Technical Reportcited by 381
  13. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
  14. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 370
  15. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 294
  16. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
  17. 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentioncited by 248
  18. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
  19. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
  20. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 226
  21. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
  22. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  23. 2024OmniParser for Pure Vision Based GUI Agentcited by 193
  24. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 183
  25. 2024Aria-UI: Visual Grounding for GUI Instructionscited by 136
  26. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 132
  27. 2024GUICourse: From General Vision Language Models to Versatile GUI Agentscited by 112
  28. 2025GTA1: GUI Test-time Scaling Agentcited by 102
  29. 2021UIBert: Learning Generic Multimodal Representations for UI Understandingcited by 93
  30. 2025UI-Venus Technical Report: Building High-performance UI Agents with RFTcited by 90
  31. 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agentscited by 89
  32. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 79
  33. 2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interactioncited by 68
  34. 2025InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflectioncited by 67
  35. 2022ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshotscited by 65
  36. 2025MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agentscited by 64
  37. 2023WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semanticscited by 45
  38. 2025Step-GUI Technical Reportcited by 38

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025MEGA-GUI: Multi-stage Enhanced Grounding Agents for GUI Elements
  2. 2025Generalist Scanner Meets Specialist Locator: A Synergistic Coarse-to-Fine Framework for Robust GUI Grounding
  3. 2026AdaZoom-GUI: Adaptive Zoom-based GUI Grounding with Instruction Refinement
  4. 2025Learning GUI Grounding with Spatial Reasoning from Visual Feedback
  5. 2024Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding
  6. 2026GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs
  7. 2026GoClick: Lightweight Element Grounding Model for Autonomous GUI Interaction
  8. 2025Zoom in, Click out: Unlocking and Evaluating the Potential of Zooming for GUI Grounding