Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Workspace learning requires AI agents to identify, reason over, exploit, and update explicit and implicit dependencies among heterogeneous files in a worker's workspace, enabling them to complete both routine and advanced tasks effectively. Despite its importance, existing relevant benchmarks largely evaluate agents on pre-specified or synthesized files with limited real-world dependencies, leaving workspace-level evaluation underexplored. To this end, we introduce Workspace-Bench, a benchmark for evaluating AI agents on Workspace Learning involving Large-Scale File Dependencies. We construct realistic workspaces with 5 worker profiles, 74 file types, 20,476 files (up to 20GB) and curate 388 tasks, each with its own file dependency graph, evaluated across 7,399 total rubrics that require cross-file retrieval, contextual reasoning, and adaptive decision-making. We further provide Workspace-Bench-Lite, a 100-task subset that preserves the benchmark distribution while reducing evaluation costs by about 70%. We evaluate 4 popular agent harnesses and 7 foundation models. Experimental results show that current agents remain far from reliable workspace learning, where the best reaches only about 60%, substantially below the human result of 80.7%, and the average performance across agents is only 43.3%.
Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 18 of its 32 references that are in the map. First posted on arXiv 5 May 2026.
Builds on
18 of 32 references.
Closest ancestors · 6
Same direction, newest first
- 2026ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents26 Apr
- 2026ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces6 Apr
- 2026OneMillion-Bench: How Far are Language Agents from Human Experts?9 Mar
- 2025Multi-Mission Tool Bench: Assessing the Robustness of LLM based Agents through Related and Dynamic Missions3 Apr
- 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks18 Dec
- 2024WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting1 May
Other references · 12
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
- 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
- 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
- 2025CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactionscited by 46
- 2026OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoningcited by 37
- 2026Odysseys: Benchmarking Web Agents on Realistic Long Horizon Taskscited by 19
- 2026DataCross: A Unified Benchmark and Agent Framework for Cross-Modal Heterogeneous Data Analysiscited by 6
Built on by
Citing papers per year: 2026: 20.
20 papers.
Sort:
By area:
- 2026Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Constructioncited by 5
- 2026The Last AI Built by Humans: Toward Genuine Recursive Self-Improvementcited by 5
- 2026OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agentscited by 1
- 2026VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?cited by 1
- 2026StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflowscited by 1
- 2026Agents in the Large: Perception-Centered Architecture for Persistent Agentscited by 1
- 2026ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End?cited by 0
- 2026AI Snitches Get Glitches: Towards Evading Agentic Surveillancecited by 0
- 2026StagedWorkspace: A Versioned Workspace for Knowledge-Work Agentscited by 0
- 2026DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflowscited by 0
- 2026Benchmarking General Mobile Assistants in Challenging Real-World Scenarioscited by 0
- 2026Atria Dawn: The Dawn of Agentic Superintelligencecited by 0
8 more
- 2026WorkWorlds: An Infrastructure for Evaluating AI Agents on Workplace Taskscited by 0
- 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvementcited by 0
- 2026HM-ROUTER: Joint Model and Harness Routing for Agentic Systemscited by 0
- 2026"You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accusedcited by 0
- 2026LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimizationcited by 0
- 2026GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesiscited by 0
- 2026VeriHarness: Scaling Agentic Verification for Long-Horizon Taskscited by 0
- 2026RunningTab: Direct Workspace Interaction with Environment-Side Tabscited by 0
- 2026RunningTab: Direct Workspace Interaction with Environment-Side Tabs7 Oct 2026
- 2026VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks1 Oct 2026
- 2026GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis30 Sep 2026
- 2026LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization28 Sep 2026
- 2026"You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accused26 Sep 2026
- 2026HM-ROUTER: Joint Model and Harness Routing for Agentic Systems26 Sep 2026
- 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvement24 Sep 2026
- 2026WorkWorlds: An Infrastructure for Evaluating AI Agents on Workplace Tasks20 Sep 2026
- 2026Atria Dawn: The Dawn of Agentic Superintelligence14 Sep 2026
- 2026The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement10 Sep 2026
- 2026Agents in the Large: Perception-Centered Architecture for Persistent Agents31 Aug 2026
- 2026DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows27 Aug 2026
8 more
- 2026Benchmarking General Mobile Assistants in Challenging Real-World Scenarios21 Aug 2026
- 2026StagedWorkspace: A Versioned Workspace for Knowledge-Work Agents18 Aug 2026
- 2026StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows18 Aug 2026
- 2026VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?11 Aug 2026
- 2026OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents4 Aug 2026
- 2026Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction23 Jul 2026
- 2026AI Snitches Get Glitches: Towards Evading Agentic Surveillance24 Jun 2026
- 2026ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End?18 Jun 2026
- 2026Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Constructioncited by 5
- 2026OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agentscited by 1
- 2026VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?cited by 1
- 2026StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflowscited by 1
- 2026Agents in the Large: Perception-Centered Architecture for Persistent Agentscited by 1
- 2026ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End?cited by 0
- 2026AI Snitches Get Glitches: Towards Evading Agentic Surveillancecited by 0
- 2026StagedWorkspace: A Versioned Workspace for Knowledge-Work Agentscited by 0
- 2026DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflowscited by 0
- 2026Atria Dawn: The Dawn of Agentic Superintelligencecited by 0
- 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvementcited by 0
- 2026HM-ROUTER: Joint Model and Harness Routing for Agentic Systemscited by 0
12 most cited of 15 All 15 (JSON)
- 2026RunningTab: Direct Workspace Interaction with Environment-Side Tabs7 Oct 2026
- 2026VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks1 Oct 2026
- 2026"You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accused26 Sep 2026
- 2026HM-ROUTER: Joint Model and Harness Routing for Agentic Systems26 Sep 2026
- 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvement24 Sep 2026
- 2026Atria Dawn: The Dawn of Agentic Superintelligence14 Sep 2026
- 2026Agents in the Large: Perception-Centered Architecture for Persistent Agents31 Aug 2026
- 2026DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows27 Aug 2026
- 2026StagedWorkspace: A Versioned Workspace for Knowledge-Work Agents18 Aug 2026
- 2026StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows18 Aug 2026
- 2026VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?11 Aug 2026
- 2026OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents4 Aug 2026
12 newest of 15 All 15 (JSON)
- 2026Benchmarking General Mobile Assistants in Challenging Real-World Scenarioscited by 0
- 2026WorkWorlds: An Infrastructure for Evaluating AI Agents on Workplace Taskscited by 0
- 2026LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimizationcited by 0
- 2026GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesiscited by 0
- 2026GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis30 Sep 2026
- 2026LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization28 Sep 2026
- 2026WorkWorlds: An Infrastructure for Evaluating AI Agents on Workplace Tasks20 Sep 2026
- 2026Benchmarking General Mobile Assistants in Challenging Real-World Scenarios21 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields
- 2026The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge
- 2026OR-Space: A Full-Lifecycle Workspace Benchmark for Industrial Optimization Agents
- 2026OS-Marathon: Benchmarking Computer-Use Agents on Vast-Horizon, Repetitive Tasks
- 2026Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation
- 2026MacAgentBench: Benchmarking AI Agents on Real-World macOS Desktop
- 2026WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents
- 2026AgentIF-OneDay: A Task-level Instruction-Following Benchmark for General AI Agents in Daily Scenarios