# CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents

arXiv: https://arxiv.org/abs/2605.25624 · HTML: https://mlsanity.com/paper/2605.25624/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use.
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 25 May 2026.

## Placed in

- LLM agents and tool use / [Reinforcement learning for LLM agents](https://mlsanity.com/agents/reinforcement-learning-for-llm-agents/)

## Builds on (most cited first, 27 of 27)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [Group Sequence Policy Optimization](https://mlsanity.com/paper/2507.18071/) (2025)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [SWE-smith: Scaling Data for Software Engineering Agents](https://mlsanity.com/paper/2504.21798/) (2025)
- [Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction](https://mlsanity.com/paper/2412.04454/) (2024)
- [UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2509.02544/) (2025)
- [OpenCUA: Open Foundations for Computer-Use Agents](https://mlsanity.com/paper/2508.09123/) (2025)
- [R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents](https://mlsanity.com/paper/2504.07164/) (2025)
- [Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis](https://mlsanity.com/paper/2505.13227/) (2025)
- [AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials](https://mlsanity.com/paper/2412.09605/) (2024)
- [SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience](https://mlsanity.com/paper/2508.04700/) (2025)
- [Endless Terminals: Scaling RL Environments for Terminal Agents](https://mlsanity.com/paper/2601.16443/) (2026)
- [EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience](https://mlsanity.com/paper/2601.15876/) (2026)
- [RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments](https://mlsanity.com/paper/2511.07317/) (2025)
- [ZeroGUI: Automating Online GUI Learning at Zero Human Cost](https://mlsanity.com/paper/2505.23762/) (2025)
- [On Data Engineering for Scaling LLM Terminal Capabilities](https://mlsanity.com/paper/2602.21193/) (2026)
- [Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning](https://mlsanity.com/paper/2508.03501/) (2025)
- [UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action](https://mlsanity.com/paper/2510.17790/) (2025)
- [InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training](https://mlsanity.com/paper/2601.04126/) (2026)
- [Gym-Anything: Turn any Software into an Agent Environment](https://mlsanity.com/paper/2604.06126/) (2026)
- [Scaling Synthetic Task Generation for Agents via Exploration](https://mlsanity.com/paper/2509.25047/) (2025)
- [VideoAgentTrek: Computer Use Pretraining from Unlabeled Videos](https://mlsanity.com/paper/2510.19488/) (2025)
- [AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machines](https://mlsanity.com/paper/2602.14296/) (2026)
- [SWE-Universe: Scale Real-World Verifiable Environments to Millions](https://mlsanity.com/paper/2602.02361/) (2026)
- [GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training](https://mlsanity.com/paper/2602.14093/) (2026)
- [SWE-Next: Scalable Real-World Software Engineering Tasks for Agents](https://mlsanity.com/paper/2603.20691/) (2026)

## Built on by (most cited first, 23 of 23)

- [OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks](https://mlsanity.com/paper/2606.29537/) (2026)
- [Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents](https://mlsanity.com/paper/2607.28227/) (2026)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Qwen-CUA: Native Computer Use for (almost) Everything](https://mlsanity.com/paper/2608.02352/) (2026)
- [StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure](https://mlsanity.com/paper/2607.11388/) (2026)
- [Fara-1.5: Scalable Learning Environments for Computer Use Agents](https://mlsanity.com/paper/2606.20785/) (2026)
- [Xiaomi-GUI-0 Technical Report](https://mlsanity.com/paper/2606.31410/) (2026)
- [SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation](https://mlsanity.com/paper/2607.05943/) (2026)
- [PhoneBuddy: Training Open Models for Agentic Phone Use](https://mlsanity.com/paper/2606.23049/) (2026)
- [EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agents](https://mlsanity.com/paper/2607.09773/) (2026)
- [Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scale](https://mlsanity.com/paper/2607.28074/) (2026)
- [CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents](https://mlsanity.com/paper/2609.05374/) (2026)
- [DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculum](https://mlsanity.com/paper/2609.19801/) (2026)
- [SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agents](https://mlsanity.com/paper/2607.23263/) (2026)
- [UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations](https://mlsanity.com/paper/2608.15930/) (2026)
- [RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents](https://mlsanity.com/paper/2609.22000/) (2026)
- [CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learning](https://mlsanity.com/paper/2609.32750/) (2026)
- [HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents](https://mlsanity.com/paper/2609.38008/) (2026)
- [cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents](https://mlsanity.com/paper/2609.40284/) (2026)
- [AutoGUIWorld: Image Generators as Visual World Models for GUI Agent](https://mlsanity.com/paper/2610.01215/) (2026)
- [Not All Experience Belongs in the Weights: Component Routing for Self-Improving GUI Agents](https://mlsanity.com/paper/2610.01787/) (2026)
- [DeskForge: Dense Supervision from Desktop Environments for Computer-Use Agents](https://mlsanity.com/paper/2610.02320/) (2026)
- [Player-1: Learning to Play from Synthetic Experience](https://mlsanity.com/paper/2610.20584/) (2026)

## Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) has driven breakthroughs in domains such as math, tool-use, and software engineering, yet its extension to computer-use agents (CUAs) has been bottlenecked by the scarcity of scalable training data with deterministic rewards. Constructing such data for CUAs requires consistent task instruction, executable environment, and verifiable reward. However, hand-curated benchmarks achieve high reward fidelity but cover few applications and LLM-as-judge-based datasets scale broadly but lack reliable verification. We present CUA-Gym, a scalable pipeline that co-generates task instructions, environment states, and reward functions. Concretely, a Generator agent constructs the initial and golden environment states, and a separate Discriminator agent writes the reward function from the task specification. An orchestrator agent drives the two through iterative rounds upon execution. Generated tuples then pass a final filter combining LLM majority voting and agent rollouts, ensuring quality beyond the per-task adversarial loop. To address the scarcity of training environments, we further synthesize CUA-Gym-Hub, a broad suite of high-fidelity mock web applications grounded in real-world software-use distributions, expanding the scale of CUA RLVR data by magnitude. Using this pipeline, we construct CUA-Gym, a dataset of 32,112 verified RLVR training tuples grounded in 110 environments. Trained with GSPO on CUA-Gym, our CUA-Gym-A3B and CUA-Gym-A17B achieve 62.1% and 72.6% on OSWorld-Verified, outperforming prior open-source CUAs at comparable scales, with performance scaling smoothly in both data volume and environment diversity. The same checkpoints also improve on the held-out WebArena benchmark, indicating transfer beyond the training environments. We will open-source the full synthesis pipeline, dataset, CUA-Gym-Hub environments, and models.