mlsanityData to 7 Oct 2026

WinDOM: Self-Family Distillation for Small-Model GUI Grounding

arXiv 2606.25964 ↗First posted 24 Jun 2026cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Small (~2B) GUI-grounding agents are attractive for on-device deployment, accessibility tooling, and low-cost iteration, but at this scale they face two open recipe questions: how to obtain bounding-box training data without expensive human annotation, and how to combine supervised fine-tuning with reinforcement learning. We address both, with the explicit goal of pushing small-model performance rather than scaling up. WinDOM is a 54,425-record grounding corpus harvested by driving an open-source Windows 11 web reimplementation under headless Playwright, with bounding boxes read directly off the DOM and no OCR or human annotation. Self-Family Distillation (SFD) is a single rejection-sampling cold-start parameterised only by the teacher choice: either an EMA of the student (no external model) or a frozen larger same-family teacher. We then treat the saturation depth of the SFD cold-start as an explicit GRPO hyperparameter. On a Qwen3.5-2B student, the under-saturated cold-start is a better GRPO initialiser than the converged one: SFD-4B with Early-init RL gains +5.4 OOD-mean (+3.5 ScreenSpot-Pro, +7.0 OSWorld-G, +5.8 ScreenSpot-V2) over the base. The same-size EMA mode lands within roughly one OOD-mean point of the cross-size 4B variant (65.2 vs 66.3) without an external teacher.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 25 references that are in the map. First posted on arXiv 24 Jun 2026.

Builds on

22 of 25 references.

References · 22

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  5. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  6. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  7. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  8. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  9. 2018Born Again Neural Networkscited by 539
  10. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 370
  11. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  12. 2026Reinforcement Learning via Self-Distillationcited by 285
10 more
  1. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
  2. 2026Self-Distillation Enables Continual Learningcited by 212
  3. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
  4. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 166
  5. 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasonerscited by 140
  6. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 132
  7. 2025GTA1: GUI Test-time Scaling Agentcited by 102
  8. 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 73
  9. 2025RL Fine-Tuning Heals OOD Forgetting in SFTcited by 27
  10. 2025Metis-SPECS: Decoupling Multimodal Learning via Self-distilled Preference-based Cold Startcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding
  2. 2025GUI-Bee: Align GUI Action Grounding to Novel Environments via Autonomous Exploration
  3. 2025VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Tasks
  4. 2026Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation
  5. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning
  6. 2026VISTA: View-Consistent Self-Verified Training for GUI Grounding
  7. 2026Trust the Right Teacher: Quality-Aware Self-Distillation for GUI Grounding
  8. 2026Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Grounding