# WebSTAR: Scalable Data Synthesis for Computer Use Agents with Step-Level Filtering

arXiv: https://arxiv.org/abs/2512.10962 · HTML: https://mlsanity.com/paper/2512.10962/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 29 references that are in the map.
- First posted on arXiv 22 Nov 2025.

## Placed in

- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)
- Synthetic data and self-improvement for LLMs / [Self-evolving LLM agents](https://mlsanity.com/synth/self-evolving-llm-agents/)

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [GPT-4V(ision) is a Generalist Web Agent, if Grounded](https://mlsanity.com/paper/2401.01614/) (2024)
- [Reinforced Self-Training (ReST) for Language Modeling](https://mlsanity.com/paper/2308.08998/) (2023)
- [Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models](https://mlsanity.com/paper/2312.06585/) (2023)
- [Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration](https://mlsanity.com/paper/1802.08802/) (2018)
- [Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction](https://mlsanity.com/paper/2412.04454/) (2024)
- [Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale](https://mlsanity.com/paper/2409.08264/) (2024)
- [UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2509.02544/) (2025)
- [DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning](https://mlsanity.com/paper/2406.11896/) (2024)
- [Agent S: An Open Agentic Framework that Uses Computers Like a Human](https://mlsanity.com/paper/2410.08164/) (2024)
- [An Illusion of Progress? Assessing the Current State of Web Agents](https://mlsanity.com/paper/2504.01382/) (2025)
- [OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis](https://mlsanity.com/paper/2412.19723/) (2024)
- [AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials](https://mlsanity.com/paper/2412.09605/) (2024)
- [GUICourse: From General Vision Language Models to Versatile GUI Agents](https://mlsanity.com/paper/2406.11317/) (2024)
- [OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimization](https://mlsanity.com/paper/2410.19609/) (2024)
- [ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy](https://mlsanity.com/paper/2403.14589/) (2024)
- [TD-Regularized Actor-Critic Methods](https://mlsanity.com/paper/1812.08288/) (2018)

## Built on by (most cited first, 11 of 11)

- [Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents](https://mlsanity.com/paper/2602.16855/) (2026)
- [GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL](https://mlsanity.com/paper/2602.22190/) (2026)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agents](https://mlsanity.com/paper/2606.02031/) (2026)
- [Scaling GUI Agents with Visual State Transitions](https://mlsanity.com/paper/2607.24112/) (2026)
- [SWE-Prime: Fewer Trajectories, Better Performance](https://mlsanity.com/paper/2608.27449/) (2026)
- [What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents](https://mlsanity.com/paper/2608.27260/) (2026)
- [ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction](https://mlsanity.com/paper/2605.11212/) (2026)
- [Learn from Weaknesses: Automated Domain Specialization for Small Computer-Use Agents](https://mlsanity.com/paper/2605.28775/) (2026)
- [PRO-CUA: Process-Reward Optimization for Computer Use Agents](https://mlsanity.com/paper/2605.29119/) (2026)
- [AutoGUIWorld: Image Generators as Visual World Models for GUI Agent](https://mlsanity.com/paper/2610.01215/) (2026)

## Abstract (arXiv)

Computer use agents (CUAs) can operate real-world digital interfaces but remain difficult to train due to the high cost of graphical user interface (GUI) interaction and the scarcity of high-quality trajectory data. Existing datasets rely on human demonstrations, limiting scalability. A natural alternative is to synthesize data from strong CUAs, yet their rollouts are highly noisy, with incorrect or suboptimal actions consisting a large proportion of the steps, making naive imitation ineffective. To tackle this challenge, we introduce a scalable data synthesis pipeline that transforms noisy rollouts into reliable supervision without human annotation. The core idea is step-level filtering, which evaluates actions individually to retain only correct steps, complemented by reasoning augmentation for improved planning. Using this pipeline, we construct WebSTAR, a dataset of 13.3K trajectories and 267K graded, reasoning-rich steps synthesized from OpenAI's computer-use-preview model. We train Qwen-2.5-VL-Instruct models (7B and 32B) on WebSTAR. On WebVoyager, our 7B model surpasses SoTA open-source CUA model UI-TARS-1.5-7B by more than 15% with only supervised finetuning. Building on step-level grading, we further create WebSCORE, a dataset of graded step-level actions, and train StepRM, a 7B multimodal process reward model distilled from o4-mini, which matches its grading quality while being far more efficient to deploy at scale. Our results establish step-level filtering as a key principle for scalable CUA training and construct two new datasets (WebSTAR, WebSCORE) and a lightweight process reward model (StepRM) as practical tools to advance robust and efficient CUAs.