mlsanityData to 14 Oct 2026

ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Data

arXiv 2509.15221 ↗First posted 18 Sep 2025cs.CVcited by 60

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Vision-Language Models (VLMs) have enabled computer use agents (CUAs) that operate GUIs autonomously, showing great potential, yet progress is limited by the lack of large-scale, open-source computer use data and foundation models. In this work, we introduce ScaleCUA, a step toward scaling open-source CUAs. It offers a large-scale dataset spanning 6 operating systems and 3 task domains, built via a closed-loop pipeline uniting automated agents with human experts. Trained on this scaled-up data, ScaleCUA can operate seamlessly across platforms. Specifically, it delivers strong gains over baselines (+26.6 on WebArena-Lite-v2, +10.7 on ScreenSpot-Pro) and sets new state-of-the-art results (94.4% on MMBench-GUI L1-Hard, 60.6% on OSWorld-G, 47.4% on WebArena-Lite-v2). These findings underscore the power of data-driven scaling for general-purpose computer use agents. We will release data, models, and code to advance future research: https://github.com/OpenGVLab/ScaleCUA.

Key facts. Cited by 60 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 65 of its 97 references that are in the map. First posted on arXiv 18 Sep 2025.

Builds on

65 of 97 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
  2. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
  3. 2025MiMo-VL Technical Report4 Jun
  4. 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners19 Apr
  5. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents14 Apr
  6. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
  7. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  8. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
  9. 2024VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents12 Aug
  10. 2024OmniParser for Pure Vision Based GUI Agent1 Aug
  11. 2024GUICourse: From General Vision Language Models to Versatile GUI Agents17 Jun
  12. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models25 Jan

Other references · 53

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  7. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  8. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  9. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  10. 2025Gemma 3 Technical Reportcited by 1,824
  11. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  12. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
38 more
  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  2. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  3. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  4. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  5. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  6. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  7. 2025Kimi-VL Technical Reportcited by 388
  8. 2025Seed1.5-VL Technical Reportcited by 373
  9. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
  10. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
  11. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 232
  12. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 223
  13. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 205
  14. 2024On the Effects of Data Scale on UI Control Agentscited by 194
  15. 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 194
  16. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 190
  17. 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 174
  18. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 168
  19. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 163
  20. 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agentscited by 159
  21. 2024Aria-UI: Visual Grounding for GUI Instructionscited by 140
  22. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 139
  23. 2024WebCanvas: Benchmarking Web Agents in Online Environmentscited by 121
  24. 2024AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agentscited by 113
  25. 2025GTA1: GUI Test-time Scaling Agentcited by 106
  26. 2025GUI-Actor: Coordinate-Free Visual Grounding for GUI Agentscited by 95
  27. 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
  28. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 81
  29. 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 74
  30. 2025MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agentscited by 67
  31. 2025ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflowscited by 52
  32. 2025ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agentscited by 50
  33. 2024VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Taskscited by 47
  34. 2025TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agentscited by 42
  35. 2025PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PCcited by 40
  36. 2025OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Modelscited by 23
  37. 2025Efficient Agent Training for Computer Usecited by 18
  38. 2025Breaking the Data Barrier -- Building GUI Agents Through Task Generalizationcited by 13

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 52.

60 papers.

Sort:

By area:

  1. 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 91
  2. 2026EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experiencecited by 53
  3. 2025Step-GUI Technical Reportcited by 39
  4. 2026LongCat-Next: Lexicalizing Modalities as Discrete Tokenscited by 36
  5. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 28
  6. 2026WebXSkill: Skill Learning for Autonomous Web Agentscited by 24
  7. 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agentcited by 21
  8. 2025OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflowscited by 19
  9. 2025Ferret-UI Lite: Lessons from Building Small On-Device GUI Agentscited by 16
  10. 2026GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RLcited by 16
  11. 2026UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agentscited by 12
  12. 2026UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experiencecited by 11
38 more
  1. 2026OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesiscited by 11
  2. 2026Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Groundingcited by 11
  3. 2026ToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agentscited by 11
  4. 2026MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agentscited by 11
  5. 2026OS-Themis: A Scalable Critic Framework for Generalist GUI Rewardscited by 10
  6. 2026TIDE: Trajectory-based Diagnostic Evaluation of Test-Time Improvement in LLM Agentscited by 7
  7. 2026MobileForge: Annotation-Free Adaptation for Mobile GUI Agents with Hierarchical Feedback-Guided Policy Optimizationcited by 7
  8. 2026StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structurecited by 7
  9. 2025OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic Modelscited by 6
  10. 2026Autonomous Continual Learning for Environment Adaptation of Computer-Use Agentscited by 6
  11. 2026OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agentscited by 6
  12. 2026Retrospective Progress-Aware Self-Refinement for LLM Agent Trainingcited by 6
  13. 2025InteractScience: Programmatic and Visually-Grounded Evaluation of Interactive Scientific Demonstration Code Generationcited by 4
  14. 2026Avenir-Web: Human-Experience-Imitating Multimodal Web Agents with Mixture of Grounding Expertscited by 4
  15. 2026TreeCUA: Efficiently Scaling GUI Automation with Tree-Structured Verifiable Evolutioncited by 4
  16. 2026UI-KOBE: Knowledge-Oriented Behavior Exploration for Lightweight Graph-Guided GUI Agentscited by 4
  17. 2026Trust the Right Teacher: Quality-Aware Self-Distillation for GUI Groundingcited by 4
  18. 2026Scaling GUI Agents with Visual State Transitionscited by 4
  19. 2026Video-Based Reward Modeling for Computer-Use Agentscited by 3
  20. 2026Mem-W: Latent Memory-Native GUI Agentscited by 3
  21. 2026Recovering Policy-Induced Errors: Benchmarking and Trajectory Synthesis for Robust GUI Agentscited by 3
  22. 2026MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Managementcited by 3
  23. 2026EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agentscited by 3
  24. 2026MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillationcited by 3
  25. 2026Screenshots or Tools? Eliciting Tool Use and Managing Multimodal Context in Hybrid GUI-MCP Computer-Use Agentscited by 3
  26. 2025Thinking by Doing: Building Efficient World Model Reasoning in LLMs via Multi-turn Interactioncited by 2
  27. 2026Scalable Behaviour Cloning on Browser Using via Skill Distillationcited by 2
  28. 2026The Next Screenshot Knows: Gated Hindsight Distillation for Mobile GUI Agentscited by 2
  29. 2025EtCon: Edit-then-Consolidate for Reliable Knowledge Editingcited by 1
  30. 2026IntentScore: Intent-Conditioned Action Evaluation for Computer-Use Agentscited by 1
  31. 2026ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy Reductioncited by 1
  32. 2026CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editingcited by 1
  33. 2026A History-Aware Visually Grounded Critic for Computer Use Agentscited by 1
  34. 2026AudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generationcited by 1
  35. 2026LabOSBench: Benchmarking Computer Use Agents for Scientific Instrument Controlcited by 1
  36. 2026UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrationscited by 1
  37. 2026Wuying-Browser-Agent: Real-World Centric Fundamental Long-Horizon Browser Agentscited by 1
  38. 2026Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignmentcited by 0

50 most cited of 60 All 60 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL
  2. 2025Scaling Agents for Computer Use
  3. 2026CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents
  4. 2026cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents
  5. 2026CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents
  6. 2025UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action
  7. 2026CUAAudit: Meta-Evaluation of Vision-Language Models as Auditors of Autonomous Computer-Use Agents
  8. 2025GUI-360^circ: A Comprehensive Dataset and Benchmark for Computer-Using Agents