WinDOM: Self-Family Distillation for Small-Model GUI Grounding
Not on the map. Why?
Abstract (arXiv)
Small (~2B) GUI-grounding agents are attractive for on-device deployment, accessibility tooling, and low-cost iteration, but at this scale they face two open recipe questions: how to obtain bounding-box training data without expensive human annotation, and how to combine supervised fine-tuning with reinforcement learning. We address both, with the explicit goal of pushing small-model performance rather than scaling up. WinDOM is a 54,425-record grounding corpus harvested by driving an open-source Windows 11 web reimplementation under headless Playwright, with bounding boxes read directly off the DOM and no OCR or human annotation. Self-Family Distillation (SFD) is a single rejection-sampling cold-start parameterised only by the teacher choice: either an EMA of the student (no external model) or a frozen larger same-family teacher. We then treat the saturation depth of the SFD cold-start as an explicit GRPO hyperparameter. On a Qwen3.5-2B student, the under-saturated cold-start is a better GRPO initialiser than the converged one: SFD-4B with Early-init RL gains +5.4 OOD-mean (+3.5 ScreenSpot-Pro, +7.0 OSWorld-G, +5.8 ScreenSpot-V2) over the base. The same-size EMA mode lands within roughly one OOD-mean point of the cross-size 4B variant (65.2 vs 66.3) without an external teacher.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 25 references that are in the map. First posted on arXiv 24 Jun 2026.
Builds on
22 of 25 references.
References · 22
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
- 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
- 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
- 2018Born Again Neural Networkscited by 539
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 370
- 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
- 2026Reinforcement Learning via Self-Distillationcited by 285
10 more
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
- 2026Self-Distillation Enables Continual Learningcited by 212
- 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
- 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 166
- 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasonerscited by 140
- 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 132
- 2025GTA1: GUI Test-time Scaling Agentcited by 102
- 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 73
- 2025RL Fine-Tuning Heals OOD Forgetting in SFTcited by 27
- 2025Metis-SPECS: Decoupling Multimodal Learning via Self-distilled Preference-based Cold Startcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding
- 2025GUI-Bee: Align GUI Action Grounding to Novel Environments via Autonomous Exploration
- 2025VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Tasks
- 2026Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation
- 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning
- 2026VISTA: View-Consistent Self-Verified Training for GUI Grounding
- 2026Trust the Right Teacher: Quality-Aware Self-Distillation for GUI Grounding
- 2026Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Grounding