Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
Recent years have witnessed the rapid evolution of AI agents toward handling increasingly complex, real-world tasks. However, existing benchmarks rarely evaluate whether agents can operate graphical user interfaces to complete long-horizon, high-value professional workflows across diverse domains. Current GUI benchmarks still predominantly focus on general-purpose software, relatively simple applications, and short-horizon tasks, leaving it largely unknown whether modern agents can follow user instructions to autonomously operate domain-specific professional software and accomplish economically valuable work in an end-to-end manner. To bridge this gap, we introduce Workflow-GYM, a benchmark for long-horizon GUI tasks centered on professional domains and specialized software environments. Through extensive experiments on state-of-the-art models, we find that even the strongest models achieve only slightly above 30% success rates, highlighting that professional long-horizon GUI workflows remain highly challenging for current GUI agents. Further analysis reveals that current agents struggle to maintain long-horizon workflow consistency, frequently exhibiting workflow stage omission, error propagation, objective drift, and insufficient understanding of professional software environments. Our findings provide important insights into the limitations of current agent systems and suggest key directions for the next generation of GUI-agent research.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 16 of its 22 references that are in the map. First posted on arXiv 9 Jun 2026.
Builds on
16 of 22 references.
Closest ancestors · 7
Same direction, newest first
- 2026AmbiBench: Benchmarking Mobile GUI Agents Beyond One-Shot Instructions in the Wild12 Feb
- 2025GUI-360^circ: A Comprehensive Dataset and Benchmark for Computer-Using Agents6 Nov
- 2025SCUBA: Salesforce Computer Use Benchmark30 Sep
- 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
- 2025PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PC20 Feb
- 2024Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents1 Jul
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr
Other references · 9
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
- 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 158
- 2025NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agentscited by 65
- 2025ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflowscited by 52
- 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 50
- 2026OneMillion-Bench: How Far are Language Agents from Human Experts?cited by 13
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026PhoneBuddy: Training Open Models for Agentic Phone Usecited by 4
- 2026OmegaUse-SOP: SOP Engineering for Professional Computer Use from Human Demonstrationscited by 1
- 2026BlueLM-GUI Technical Report: A Real-Device-Centric Flywheel for Self-Improving Mobile GUI Agentscited by 0
- 2026AutoGUIWorld: Image Generators as Visual World Models for GUI Agentcited by 0
- 2026MiniCorp: The Last Mile to an AI Agent Corporationcited by 0
- 2026MiniCorp: The Last Mile to an AI Agent Corporation5 Oct 2026
- 2026AutoGUIWorld: Image Generators as Visual World Models for GUI Agent1 Oct 2026
- 2026BlueLM-GUI Technical Report: A Real-Device-Centric Flywheel for Self-Improving Mobile GUI Agents11 Sep 2026
- 2026OmegaUse-SOP: SOP Engineering for Professional Computer Use from Human Demonstrations2 Sep 2026
- 2026PhoneBuddy: Training Open Models for Agentic Phone Use22 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026WindowsWorld: A Process-Centric Benchmark of Autonomous GUI Agents in Professional Cross-Application Environments
- 2026DeskCraft: Benchmarking Desktop Agents on Professional Workflows and Human-in-the-Loop Collaboration
- 2026LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces
- 2026The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge
- 2026SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition
- 2026Engineering Robustness into Personal Agents with the AI Workflow Store
- 2026FlowSteer: Towards Agents Designing Agentic Workflows via Reinforced Progressive Canvas Editing
- 2025Assistance or Disruption? Exploring and Evaluating the Design and Trade-offs of Proactive AI Programming Support