OpenComputer: Verifiable Software Worlds for Computer-Use Agents
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
We present OpenComputer, a verifier-grounded framework for constructing verifiable software worlds for computer-use agents. OpenComputer integrates four components: (1) app-specific state verifiers that expose structured inspection endpoints over real applications, (2) a self-evolving verification layer that improves verifier reliability using execution-grounded feedback, (3) a task-generation pipeline that synthesizes realistic and machine-checkable desktop tasks, and (4) an evaluation harness that records full trajectories and computes auditable partial-credit rewards. In its current form, OpenComputer covers 33 desktop applications and 1,000 finalized tasks spanning browsers, office tools, creative software, development environments, file managers, and communication applications. Experiments show that OpenComputer's hard-coded verifiers align more closely with human adjudication than LLM-as-judge evaluation, especially when success depends on fine-grained application state. Frontier agents struggle with end-to-end completion despite partial progress, and open-source models exhibit sharp drops from their OSWorld-Verified scores, exposing a persistent gap in robust computer automation.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 39 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
32 of 39 references.
Closest ancestors · 10
Same direction, newest first
- 2026Gym-Anything: Turn any Software into an Agent Environment7 Apr
- 2026CUAAudit: Meta-Evaluation of Vision-Language Models as Auditors of Autonomous Computer-Use Agents11 Mar
- 2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents15 Feb
- 2026GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training15 Feb
- 2025SCUBA: Salesforce Computer Use Benchmark30 Sep
- 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents1 Apr
- 2024GUI Agents: A Survey18 Dec
- 2024Agent S: An Open Agentic Framework that Uses Computers Like a Human10 Oct
- 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale12 Sep
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr
Other references · 22
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
- 2023Large Language Models are not Fair Evaluatorscited by 1,050
- 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
- 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
- 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judgescited by 197
- 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
10 more
- 2025Simulating Environments with Reasoning Models for Agent Trainingcited by 52
- 2025Towards General Agentic Intelligence via Environment Scalingcited by 51
- 2026EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experiencecited by 51
- 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learningcited by 51
- 2025BEARCUBS: A benchmark for computer-using web agentscited by 29
- 2026InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Trainingcited by 26
- 2026TermiGen: High-Fidelity Environment and Robust Trajectory Synthesis for Terminal Agentscited by 24
- 2025RealWebAssist: A Benchmark for Long-Horizon Web Assistance with Real-World Userscited by 21
- 2026Immersion in the GitHub Universe: Scaling Coding Agents to Masterycited by 20
- 2024PC Agent: While You Sleep, AI Works -- A Cognitive Journey into Digital Worldcited by 12
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 6
- 2026PhoneBuddy: Training Open Models for Agentic Phone Usecited by 4
- 2026StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structurecited by 4
- 2026GUI vs. CLI: Execution Bottlenecks in Screen-Only and Skill-Mediated Computer-Use Agentscited by 3
- 2026StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agentscited by 2
- 2026Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial Environmentscited by 1
- 2026GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environmentscited by 1
- 2026DevicesWorld: Benchmarking Cross-Device Agents in Heterogeneous Environmentscited by 0
- 2026Absorbed in Inertia: Activation Analysis for Computer-Use Agentscited by 0
- 2026Absorbed in Inertia: Activation Analysis for Computer-Use Agents29 Sep 2026
- 2026GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments30 Aug 2026
- 2026Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial Environments25 Aug 2026
- 2026StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents24 Jul 2026
- 2026DevicesWorld: Benchmarking Cross-Device Agents in Heterogeneous Environments15 Jul 2026
- 2026StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure13 Jul 2026
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap29 Jun 2026
- 2026GUI vs. CLI: Execution Bottlenecks in Screen-Only and Skill-Mediated Computer-Use Agents22 Jun 2026
- 2026PhoneBuddy: Training Open Models for Agentic Phone Use22 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026The Art of Building Verifiers for Computer Use Agents
- 2026Skills as Verifiable Artifacts: A Trust Schema and a Biconditional Correctness Criterion for Human-in-the-Loop Agent Runtimes
- 2026RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents
- 2026AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions
- 2026Open, Reliable, and Collective: A Community-Driven Framework for Tool-Using AI Agents
- 2026MANTRA: Synthesizing SMT-Validated Compliance Benchmarks for Tool-Using LLM Agents
- 2026ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software
- 2026The reach of a verification tool decides its value: A controlled study of verification surface, artifact quality, and cost in AI coding agents