mlsanityData to 14 Oct 2026

CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents

arXiv 2609.05374 ↗First posted 4 Sep 2026cs.AIcited by 2

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

Computer-use agents have advanced on benchmarks like OSWorld and AndroidWorld, but still act mostly through the GUI, often producing inefficient trajectories. Real-world computer work is hybrid, combining visual-state inspection with precise, high-throughput command-line operations, so capable agents must coordinate both modalities over shared application state. Yet scalable hybrid environments remain scarce because supporting both GUI and CLI over real applications typically requires substantial manual engineering for each application. Existing agents also struggle to use the two interfaces complementarily: CLI-native agents lack visual perception for tasks involving interface state or layout, while GUI-native agents are inefficient for operations better executed through commands. We introduce CUA-Universe, a scalable environment-to-data pipeline that turns real desktop software into hybrid GUI+CLI environments. App-Forge adapts applications into reproducible VMs and command-line surfaces it discovers, wraps, or generates, scaling to 16 applications; Task-Weave synthesizes diverse hybrid tasks of controllable difficulty from reusable operations over seed files; and Path-Steer steers rollouts along efficient hybrid paths and harvests verified trajectories for post-training. Training on this data shifts behavior from inefficient GUI interaction and brittle CLI scripting toward effective GUI+CLI orchestration. Our 9B model improves both success and efficiency on CUA-Verse (Score +39.3 pts; -37% steps, -60% tokens), OSWorld (SR +16.8 pts; -57% steps, -44% tokens), and OSWorld-MCP (Score +7.84 pts; -27% steps, -30% tokens). CUA-Universe provides a scalable path toward more capable and efficient computer-use agents.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 22 of its 41 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

22 of 41 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks28 Jun
  2. 2026GUI vs. CLI: Execution Bottlenecks in Screen-Only and Skill-Mediated Computer-Use Agents22 Jun
  3. 2026MacAgentBench: Benchmarking AI Agents on Real-World macOS Desktop21 Jun
  4. 2026CLI-Anything: Towards Agent-Native Computer Use2 Jun
  5. 2026Gym-Anything: Turn any Software into an Agent Environment7 Apr
  6. 2026GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training15 Feb
  7. 2025OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agents28 Oct
  8. 2025UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action20 Oct
  9. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr

Other references · 13

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  3. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  4. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  5. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
  6. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  7. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  8. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
  9. 2026Tmax: A simple recipe for terminal agentscited by 33
  10. 2026InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Trainingcited by 27
  11. 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agentscited by 23
  12. 2026Terminal-World: Scaling Terminal-Agent Environments via Agent Skillscited by 20
1 more
  1. 2026CLI-Universe: Towards Verifiable Task Synthesis Engine for Terminal Agentscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
  2. 2026Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents
  3. 2025UFO2: The Desktop AgentOS
  4. 2026JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
  5. 2026TreeCUA: Efficiently Scaling GUI Automation with Tree-Structured Verifiable Evolution
  6. 2026ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents
  7. 2026CUAWright: A Minimal Unified Interface for Digital Agents
  8. 2025MCPWorld: A Unified Benchmarking Testbed for API, GUI, and Hybrid Computer Use Agents