mlsanityData to 7 Oct 2026

RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

arXiv 2609.22000 ↗First posted 18 Sep 2026cs.CL cs.SEcited by 1

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

Computer-use agents (CUAs) have advanced along two separate lines: graphical interaction and software development through code and the command line. Real digital work requires both, interleaved rather than stacked end to end. We study hybrid CUAs that autonomously decide when to explore an interface, implement software, and run and visually verify their artifacts. We introduce RecreationWorld, a five-platform framework built around recreation: given a running reference, an agent must discover its behavior and build a faithful implementation with no prescribed workflow. RecreationWorld provides reproducible environments on Ubuntu, macOS, Windows, Android, and Web, plus a unified harness with native GUI control and coding tools. The running reference serves as an oracle for hidden behavioral tests, providing execution-grounded rewards. We scale trajectory generation with high-quality open-source applications. Models trained on these trajectories improve across five out-of-distribution coding and hybrid computer-use benchmarks and more frequently verify their rendered outputs, providing evidence of transfer beyond recreation. For held-out evaluation, we introduce RecreationBench, comprising 250 diverse tasks across domains and platforms. Reference-grounded programmatic and visual assertions cover action-conditioned outcomes at multiple interaction depths; each is validated on the reference and by human reviewers before the suite is frozen for automatic scoring. GPT-6 Astra leads at 58.1% overall, but passes all programmatic tests on just 2.8% of tasks. Agents reproduce static interface structure more reliably than interactions and computed outputs, while generated applications remain smaller and more monolithic than their references. We release the benchmark, environments, and test suites.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 28 of its 41 references that are in the map. First posted on arXiv 18 Sep 2026.

Builds on

28 of 41 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents30 Jul
  2. 2026SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL13 Jul
  3. 2026WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces8 Jun
  4. 2026Gym-Anything: Turn any Software into an Agent Environment7 Apr
  5. 2026GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training15 Feb
  6. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
  7. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct

Other references · 21

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  2. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  3. 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 196
  4. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 146
  5. 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMscited by 73
  6. 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratchcited by 66
  7. 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototypingcited by 50
  8. 2026ProgramBench: Can Language Models Rebuild Programs From Scratch?cited by 47
  9. 2024WebCode2M: A Real-World Dataset for Code Generation from Webpage Designscited by 43
  10. 2025DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generationcited by 41
  11. 2026InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Trainingcited by 26
  12. 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agentscited by 21
9 more
  1. 2026AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machinescited by 20
  2. 2026Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verificationcited by 16
  3. 2025You Don't Know Until You Click:Automated GUI Testing for Production-Ready Software Evaluationcited by 11
  4. 2026GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?cited by 10
  5. 2026Safe and Scalable Web Agent Learning via Recreated Websitescited by 9
  6. 2026PhoneHarness: Harnessing Phone-Use Agents through Mixed GUI, CLI, and Tool Actionscited by 6
  7. 2026Qwen-CUA: Native Computer Use for (almost) Everythingcited by 5
  8. 2025AppForge: From Assistant to Independent Developer -- Are GPTs Ready for Software Development?cited by 4
  9. 2026StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agentscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Computer Use at the Edge of the Statistical Precipice
  2. 2025UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action
  3. 2026ProgramDistill: From Interactive Web Apps to Verifiable Reference-Guided SWE Tasks
  4. 2026OpenComputer: Verifiable Software Worlds for Computer-Use Agents
  5. 2025MCPWorld: A Unified Benchmarking Testbed for API, GUI, and Hybrid Computer Use Agents
  6. 2026CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents
  7. 2026Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization
  8. 2026Step-level Optimization for Efficient Computer-use Agents