mlsanityData to 14 Oct 2026

WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks

arXiv 2407.05291 ↗First posted 7 Jul 2024cs.AIcited by 90

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

The ability of large language models (LLMs) to mimic human-like intelligence has led to a surge in LLM-based autonomous agents. Though recent LLMs seem capable of planning and reasoning given user instructions, their effectiveness in applying these capabilities for autonomous task solving remains underexplored. This is especially true in enterprise settings, where automated agents hold the promise of a high impact. To fill this gap, we propose WorkArena++, a novel benchmark consisting of 682 tasks corresponding to realistic workflows routinely performed by knowledge workers. WorkArena++ is designed to evaluate the planning, problem-solving, logical/arithmetic reasoning, retrieval, and contextual understanding abilities of web agents. Our empirical studies across state-of-the-art LLMs and vision-language models (VLMs), as well as human workers, reveal several challenges for such models to serve as useful assistants in the workplace. In addition to the benchmark, we provide a mechanism to effortlessly generate thousands of ground-truth observation/action traces, which can be used for fine-tuning existing models. Overall, we expect this work to serve as a useful resource to help the community progress toward capable autonomous agents. The benchmark can be found at https://github.com/ServiceNow/WorkArena.

Key facts. Cited by 90 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 8 of its 33 references that are in the map. First posted on arXiv 7 Jul 2024.

Builds on

8 of 33 references.

References · 8

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  5. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 438
  6. 2023Language Models can Solve Computer Taskscited by 372
  7. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 299
  8. 2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 180

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 26; 2026: 61.

90 papers.

Sort:

By area:

  1. 2025Survey on Evaluation of LLM-based Agentscited by 204
  2. 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agentscited by 133
  3. 2025AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectoriescited by 108
  4. 2024GUI Agents: A Surveycited by 103
  5. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  6. 2025SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agentscited by 71
  7. 2025OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflowscited by 57
  8. 2025CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactionscited by 47
  9. 2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agentscited by 46
  10. 2026ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspacescited by 33
  11. 2025Reliable Weak-to-Strong Monitoring of LLM Agentscited by 31
  12. 2025WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Taskscited by 30
38 more
  1. 2025Benchmarking Deep Search over Heterogeneous Enterprise Datacited by 23
  2. 2026LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleaguescited by 23
  3. 2024AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrationscited by 19
  4. 2026MolmoWeb: Open Visual Web Agent and Open Data for the Open Webcited by 19
  5. 2025Build the web for agents, not agents for the webcited by 18
  6. 2025MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Themcited by 16
  7. 2025WALT: Web Agents that Learn Toolscited by 16
  8. 2025WAREX: Web Agent Reliability Evaluation on Existing Benchmarkscited by 15
  9. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 14
  10. 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Surveycited by 13
  11. 2026ContractSkill: Repairable Contract-Based Skills for Multimodal Web Agentscited by 13
  12. 2025OpenApps: Simulating Environment Variations to Measure UI-Agent Reliabilitycited by 12
  13. 2026World of Workflows: A Benchmark for Bringing World Models to Enterprise Systemscited by 11
  14. 2025EconWebArena: Benchmarking Autonomous Agents on Economic Tasks in Realistic Web Environmentscited by 9
  15. 2026EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environmentscited by 9
  16. 2025Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safetycited by 8
  17. 2025PLANET: A Collection of Benchmarks for Evaluating LLMs' Planning Capabilitiescited by 7
  18. 2025SCUBA: Salesforce Computer Use Benchmarkcited by 7
  19. 2026WebArbiter: A Principle-Guided Reasoning Process Reward Model for Web Agentscited by 7
  20. 2025FieldWorkArena: Agentic AI Benchmark for Real Field Work Taskscited by 6
  21. 2025Agentic Services Computingcited by 6
  22. 2026Paying Less Generalization Tax: A Cross-Domain Generalization Study of RL Training for LLM Agentscited by 6
  23. 2026Terminal Agents Suffice for Enterprise Automationcited by 6
  24. 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 6
  25. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  26. 2026PhoneHarness: Harnessing Phone-Use Agents through Mixed GUI, CLI, and Tool Actionscited by 6
  27. 2026Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agentscited by 6
  28. 2025JEF-Hinter: Leveraging Offline Knowledge for Improving Web Agents Adaptationcited by 5
  29. 2025Permission Manifests for Web Agentscited by 5
  30. 2026WebForge: Breaking the Realism-Reproducibility-Scalability Trilemma in Browser Agent Benchmarkcited by 5
  31. 2026SaaS-Bench: Can Computer-Use Agents Leverage Real-World SaaS to Solve Professional Workflows?cited by 4
  32. 2026Where Did It Go Wrong? Process-Level Evaluation of Web Agents with Semantic State Trackingcited by 4
  33. 2026How Benchmarks Mis-Score Computer-Use Agentscited by 4
  34. 2026AgentCE-Bench: Agent Configurable Evaluation with Scalable Horizons and Controllable Difficulty under Lightweight Environmentscited by 3
  35. 2026Structured Distillation of Web Agent Capabilities Enables Generalizationcited by 3
  36. 2026One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflowscited by 3
  37. 2026The Era by Eon Benchmark: A Generated Enterprise Estate with Exact Ground Truth for Benchmarking LLM Agentscited by 3
  38. 2026Do Enterprise Systems Need Learned World Models? The Importance of Context to Infer Dynamicscited by 2

50 most cited of 90 All 90 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?
  2. 2026PolyWorkBench: Benchmarking LLM Agents for Cross-Lingual Long-Horizon Workflows
  3. 2026ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents
  4. 2025PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literature
  5. 2026The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge
  6. 2026Characterizing Large Language Model Agentic Workflows: A Study on N8n Ecosystem
  7. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  8. 2025OmniNova:A General Multimodal Agent Framework