mlsanityData to 7 Oct 2026

WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?

arXiv 2403.07718 ↗First posted 12 Mar 2024cs.LG cs.AIcited by 348

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

We study the use of large language model-based agents for interacting with software via web browsers. Unlike prior work, we focus on measuring the agents' ability to perform tasks that span the typical daily work of knowledge workers utilizing enterprise software systems. To this end, we propose WorkArena, a remote-hosted benchmark of 33 tasks based on the widely-used ServiceNow platform. We also introduce BrowserGym, an environment for the design and evaluation of such agents, offering a rich set of actions as well as multimodal observations. Our empirical evaluation reveals that while current agents show promise on WorkArena, there remains a considerable gap towards achieving full task automation. Notably, our analysis uncovers a significant performance disparity between open and closed-source LLMs, highlighting a critical area for future exploration and development in the field.

Key facts. Cited by 348 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 32 references that are in the map. First posted on arXiv 12 Mar 2024.

Builds on

23 of 32 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models25 Jan
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul
  3. 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun
  4. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul

Other references · 19

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2022Emergent Abilities of Large Language Modelscited by 2,483
  6. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  7. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  8. 2023Language Models can Solve Computer Taskscited by 371
  9. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  10. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  11. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
  12. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 291
7 more
  1. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
  2. 2020Mapping Natural Language Instructions to Mobile UI Action Sequencescited by 180
  3. 2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 177
  4. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 159
  5. 2022A data-driven approach for learning to control computerscited by 87
  6. 2023BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agentscited by 83
  7. 2023Automatic Macro Mining from Interaction Traces at Scalecited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 33; 2025: 89; 2026: 226.

348 papers.

Sort:

By area:

  1. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  2. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  3. 2024Agent Workflow Memorycited by 342
  4. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306
  5. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 212
  6. 2024AI Agents That Mattercited by 199
  7. 2025Survey on Evaluation of LLM-based Agentscited by 190
  8. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  9. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 155
  10. 2024The BrowserGym Ecosystem for Web Agent Researchcited by 152
  11. 2025OpenCUA: Open Foundations for Computer-Use Agentscited by 151
  12. 2024Tree Search for Language Model Agentscited by 148
38 more
  1. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 132
  2. 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agentscited by 126
  3. 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
  4. 2024WebCanvas: Benchmarking Web Agents in Online Environmentscited by 119
  5. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 118
  6. 2024Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigationcited by 116
  7. 2025Inducing Programmatic Skills for Agentic Taskscited by 103
  8. 2024VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agentscited by 101
  9. 2024GUI Agents: A Surveycited by 101
  10. 2025AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectoriescited by 99
  11. 2025SafeArena: Evaluating the Safety of Autonomous Web Agentscited by 97
  12. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  13. 2024Beyond Browsing: API-Based Web Agentscited by 77
  14. 2024CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environmentscited by 69
  15. 2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interactioncited by 68
  16. 2024Refusal-Trained LLMs Are Easily Jailbroken As Browser Agentscited by 61
  17. 2024Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaborationcited by 60
  18. 2024Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?cited by 55
  19. 2024NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wildcited by 55
  20. 2025OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflowscited by 54
  21. 2026AgentRx: Diagnosing AI Agent Failures from Execution Trajectoriescited by 53
  22. 2024WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Settingcited by 47
  23. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  24. 2025CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactionscited by 46
  25. 2025Contextual Experience Replay for Self-Improvement of Language Agentscited by 46
  26. 2025DABstep: Data Agent Benchmark for Multi-step Reasoningcited by 44
  27. 2024VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Taskscited by 43
  28. 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
  29. 2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agentscited by 41
  30. 2026Code as Agent Harnesscited by 40
  31. 2024ScribeAgent: Towards Specialized Web Agents Using Production-Scale Workflow Datacited by 39
  32. 2025MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineeringcited by 38
  33. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  34. 2025CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web Navigationcited by 35
  35. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  36. 2025Generative to Agentic AI: Survey, Conceptualization, and Challengescited by 32
  37. 2025Go-Browse: Training Web Agents with Structured Explorationcited by 32
  38. 2025InSTA: Towards Internet-Scale Training For Agentscited by 31

50 most cited of 348 All 348 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks
  2. 2025An Illusion of Progress? Assessing the Current State of Web Agents
  3. 2025BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks
  4. 2025OmniNova:A General Multimodal Agent Framework
  5. 2024Unveiling Disparities in Web Task Handling Between Human and Web Agent
  6. 2026ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents
  7. 2025API Agents vs. GUI Agents: Divergence and Convergence
  8. 2026PolyWorkBench: Benchmarking LLM Agents for Cross-Lingual Long-Horizon Workflows