ZonUI-3B: A Lightweight Vision-Language Model for Cross-Resolution GUI Grounding
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
In this paper, we present ZonUI-3B, a lightweight Vision-Language Model (VLM) that can be fully trained on a single consumer-grade GPU (RTX 4090) while delivering performance comparable to significantly larger models on GUI grounding tasks. The model incorporates several key innovations: (i) combine cross-platform, multi-resolution dataset of 24K examples from diverse sources including mobile, desktop, and web GUI screenshots to effectively address data scarcity in high-resolution desktop environments; (ii) a two-stage fine-tuning strategy, where initial cross-platform training establishes robust GUI understanding, followed by specialized fine-tuning on high-resolution data to significantly enhance model adaptability; and (iii) data curation and redundancy reduction strategies, demonstrating that randomly sampling a smaller subset with reduced redundancy achieves performance comparable to larger datasets, emphasizing data diversity over sheer volume. Empirical evaluation on standard GUI grounding benchmarks, including ScreenSpot, ScreenSpot-v2, and the challenging ScreenSpot-Pro, highlights ZonUI-3B's exceptional accuracy, achieving 84.9% on ScreenSpot and 86.4% on ScreenSpot-v2, surpassing prior models under 4B parameters. Ablation studies validate the critical role of balanced sampling and two-stage fine-tuning in enhancing robustness, particularly in high-resolution desktop scenarios. The ZonUI-3B is available at: https://github.com/Han1018/ZonUI-3B
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 12 of its 12 references that are in the map. First posted on arXiv 30 Jun 2025.
Builds on
12 of 12 references.
Closest ancestors · 4
Same direction, newest first
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
- 2023CogAgent: A Visual Language Model for GUI Agents14 Dec
Other references · 8
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
- 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 232
- 2024AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agentscited by 113
Built on by
Citing papers per year: 2025: 3; 2026: 4.
7 papers.
- 2025InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy Optimizationcited by 27
- 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoningcited by 22
- 2025Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agentcited by 11
- 2026Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elementscited by 5
- 2026Avenir-Web: Human-Experience-Imitating Multimodal Web Agents with Mixture of Grounding Expertscited by 4
- 2026LiteGUI: Distilling Compact GUI Agents with Reinforcement Learningcited by 4
- 2026Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Groundingcited by 0
- 2026Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Grounding14 Sep 2026
- 2026LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning8 May 2026
- 2026Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elements15 Mar 2026
- 2026Avenir-Web: Human-Experience-Imitating Multimodal Web Agents with Mixture of Grounding Experts2 Feb 2026
- 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoning23 Oct 2025
- 2025InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy Optimization7 Aug 2025
- 2025Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent6 Aug 2025
Similar, not cited
Close in wording, no citation either way
- 2025MEGA-GUI: Multi-stage Enhanced Grounding Agents for GUI Elements
- 2025textscGUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Grounding
- 2026GoClick: Lightweight Element Grounding Model for Autonomous GUI Interaction
- 2025MVP: Multiple View Prediction Improves GUI Grounding
- 2026GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs
- 2026AdaZoom-GUI: Adaptive Zoom-based GUI Grounding with Instruction Refinement
- 2025DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoning
- 2026POINTS-GUI-G: GUI-Grounding Journey