Gym-Anything: Turn any Software into an Agent Environment
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
Computer-use agents hold the promise of assisting in a wide range of digital economic activities. However, current research has largely focused on short-horizon tasks over a limited set of software with limited economic value, such as basic e-commerce and OS-configuration tasks. A key reason is that creating environments for complex software requires significant time and human effort, and therefore does not scale. To address this, we introduce Gym-Anything, a framework for converting any software into an interactive computer-use environment. We frame environment creation itself as a multi-agent task: a coding agent writes setup scripts, downloads real-world data, and configures the software, while producing evidence of correct setup. An independent audit agent then verifies evidence for the environment setup against a quality checklist. Using a taxonomy of economically valuable occupations grounded in U.S. GDP data, we apply this pipeline to 200 software applications with broad occupational coverage. The result is CUA-World, a collection of over 10K long-horizon tasks spanning domains from medical science and astronomy to engineering and enterprise systems, each configured with realistic data along with train and test splits. CUA-World also includes CUA-World-Long, a challenging long-horizon benchmark with tasks often requiring over 500 steps, far exceeding existing benchmarks. Distilling successful trajectories from the training split into a 2B vision-language model outperforms models 2× its size. We also apply the same auditing principle at test time: a separate VLM reviews completed trajectories and provides feedback on what remains, improving Gemini-3-Flash on CUA-World-Long from 11.5% to 14.0%. We release all code, infrastructure, and benchmark data to facilitate future research in realistic computer-use agents.
Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 44 of its 65 references that are in the map. First posted on arXiv 7 Apr 2026.
Builds on
44 of 65 references.
Closest ancestors · 7
Same direction, newest first
- 2026GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training15 Feb
- 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
- 2025Efficient Agent Training for Computer Use20 May
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
- 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis27 Dec
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
- 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning14 Jun
Other references · 37
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2025Qwen3 Technical Reportcited by 8,195
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2016OpenAI Gymcited by 1,684
- 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
- 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
- 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
25 more
- 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
- 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
- 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 223
- 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 211
- 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 174
- 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
- 2024Autonomous Evaluation and Refinement of Digital Agentscited by 136
- 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
- 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 88
- 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generationcited by 80
- 2025Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agentscited by 68
- 2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experiencecited by 68
- 2025The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMscited by 62
- 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learningcited by 51
- 2024Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agentscited by 50
- 2025ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflowscited by 47
- 2024Gymnasium: A Standard Interface for Reinforcement Learning Environmentscited by 35
- 2024BAGEL: Bootstrapping Agents by Guiding Exploration with Languagecited by 34
- 2025EvilGenie: A Reward Hacking Benchmarkcited by 24
- 2025AutoEnv: Automated Environments for Measuring Cross-Environment Agent Learningcited by 18
- 2025Training Versatile Coding Agents in Synthetic Environmentscited by 13
- 2024PC Agent: While You Sleep, AI Works -- A Cognitive Journey into Digital Worldcited by 12
- 2026How Well Does Agent Development Reflect Real-World Work?cited by 10
- 2025Score the Steps, Not Just the Goal: VLM-Based Subgoal Evaluation for Robotic Manipulationcited by 8
- 2025Programming with Pixels: Can Computer-Use Agents do Software Engineering?cited by 4
Built on by
Citing papers per year: 2026: 24.
24 papers.
Sort:
By area:
- 2026Agents' Last Examcited by 25
- 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agentscited by 21
- 2026OpenComputer: Verifiable Software Worlds for Computer-Use Agentscited by 9
- 2026Fara-1.5: Scalable Learning Environments for Computer Use Agentscited by 5
- 2026Qwen-CUA: Native Computer Use for (almost) Everythingcited by 5
- 2026ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agentscited by 4
- 2026PhoneBuddy: Training Open Models for Agentic Phone Usecited by 4
- 2026VISUALSKILL: Multimodal Skills for Computer-Use Agentscited by 3
- 2026SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulationcited by 3
- 2026Multi-Agent Computer Usecited by 2
- 2026Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learningcited by 2
- 2026Claw-Anything: Benchmarking Always-On Personal Assistants with Broader Access to User's Digital Worldcited by 1
12 more
- 2026Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scalecited by 1
- 2026CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agentscited by 1
- 2026RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agentscited by 1
- 2026Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliabilitycited by 0
- 2026SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agentscited by 0
- 2026Efficient Test-Time Adaptation through Human-AI Interactioncited by 0
- 2026Report of the 8th LSVOS Challenge: Complex and Multimodal Video Object Segmentationcited by 0
- 2026Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Taskscited by 0
- 2026CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learningcited by 0
- 2026CompoWorld: Compositional Environment Scaling for General Agentscited by 0
- 2026cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agentscited by 0
- 2026AutoGUIWorld: Image Generators as Visual World Models for GUI Agentcited by 0
- 2026AutoGUIWorld: Image Generators as Visual World Models for GUI Agent1 Oct 2026
- 2026cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents30 Sep 2026
- 2026CompoWorld: Compositional Environment Scaling for General Agents27 Sep 2026
- 2026CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learning26 Sep 2026
- 2026RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents18 Sep 2026
- 2026Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks8 Sep 2026
- 2026Report of the 8th LSVOS Challenge: Complex and Multimodal Video Object Segmentation5 Sep 2026
- 2026CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents4 Sep 2026
- 2026Efficient Test-Time Adaptation through Human-AI Interaction3 Sep 2026
- 2026Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learning4 Aug 2026
- 2026Qwen-CUA: Native Computer Use for (almost) Everything3 Aug 2026
- 2026Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scale30 Jul 2026
12 more
- 2026SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agents25 Jul 2026
- 2026SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation7 Jul 2026
- 2026PhoneBuddy: Training Open Models for Agentic Phone Use22 Jun 2026
- 2026Fara-1.5: Scalable Learning Environments for Computer Use Agents18 Jun 2026
- 2026VISUALSKILL: Multimodal Skills for Computer-Use Agents16 Jun 2026
- 2026Agents' Last Exam3 Jun 2026
- 2026Multi-Agent Computer Use1 Jun 2026
- 2026Claw-Anything: Benchmarking Always-On Personal Assistants with Broader Access to User's Digital World25 May 2026
- 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents25 May 2026
- 2026OpenComputer: Verifiable Software Worlds for Computer-Use Agents19 May 2026
- 2026ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agents15 May 2026
- 2026Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability8 May 2026
- 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agentscited by 21
- 2026OpenComputer: Verifiable Software Worlds for Computer-Use Agentscited by 9
- 2026Fara-1.5: Scalable Learning Environments for Computer Use Agentscited by 5
- 2026ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agentscited by 4
- 2026PhoneBuddy: Training Open Models for Agentic Phone Usecited by 4
- 2026VISUALSKILL: Multimodal Skills for Computer-Use Agentscited by 3
- 2026Multi-Agent Computer Usecited by 2
- 2026Claw-Anything: Benchmarking Always-On Personal Assistants with Broader Access to User's Digital Worldcited by 1
- 2026Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scalecited by 1
- 2026CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agentscited by 1
- 2026RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agentscited by 1
- 2026Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliabilitycited by 0
12 most cited of 16 All 16 (JSON)
- 2026cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents30 Sep 2026
- 2026CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learning26 Sep 2026
- 2026RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents18 Sep 2026
- 2026Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks8 Sep 2026
- 2026CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents4 Sep 2026
- 2026Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scale30 Jul 2026
- 2026SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agents25 Jul 2026
- 2026PhoneBuddy: Training Open Models for Agentic Phone Use22 Jun 2026
- 2026Fara-1.5: Scalable Learning Environments for Computer Use Agents18 Jun 2026
- 2026VISUALSKILL: Multimodal Skills for Computer-Use Agents16 Jun 2026
- 2026Multi-Agent Computer Use1 Jun 2026
- 2026Claw-Anything: Benchmarking Always-On Personal Assistants with Broader Access to User's Digital World25 May 2026
12 newest of 16 All 16 (JSON)
- 2026Agents' Last Examcited by 25
- 2026Qwen-CUA: Native Computer Use for (almost) Everythingcited by 5
- 2026Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learningcited by 2
- 2026Efficient Test-Time Adaptation through Human-AI Interactioncited by 0
- 2026Report of the 8th LSVOS Challenge: Complex and Multimodal Video Object Segmentationcited by 0
- 2026CompoWorld: Compositional Environment Scaling for General Agentscited by 0
- 2026CompoWorld: Compositional Environment Scaling for General Agents27 Sep 2026
- 2026Report of the 8th LSVOS Challenge: Complex and Multimodal Video Object Segmentation5 Sep 2026
- 2026Efficient Test-Time Adaptation through Human-AI Interaction3 Sep 2026
- 2026Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learning4 Aug 2026
- 2026Qwen-CUA: Native Computer Use for (almost) Everything3 Aug 2026
- 2026Agents' Last Exam3 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026Computer Use at the Edge of the Statistical Precipice
- 2026SaaS-Bench: Can Computer-Use Agents Leverage Real-World SaaS to Solve Professional Workflows?
- 2026C-World: A Computer Use Agent Environment Creator
- 2026EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessions
- 2026CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering
- 2026CUA-Skill: Develop Skills for Computer Using Agent
- 2026Zero2Repo: Can Coding Agents Build Repositories from Scratch?
- 2026ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software