mlsanityData to 7 Oct 2026

WebArena: A Realistic Web Environment for Building Autonomous Agents

arXiv 2307.13854 ↗First posted 25 Jul 2023cs.AI cs.CL cs.LGcited by 1,908

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

With advances in generative AI, there is now potential for autonomous agents to manage daily tasks via natural language commands. However, current agents are primarily created and tested in simplified synthetic environments, leading to a disconnect with real-world scenarios. In this paper, we build an environment for language-guided agents that is highly realistic and reproducible. Specifically, we focus on agents that perform tasks on the web, and create an environment with fully functional websites from four common domains: e-commerce, social forum discussions, collaborative software development, and content management. Our environment is enriched with tools (e.g., a map) and external knowledge bases (e.g., user manuals) to encourage human-like task-solving. Building upon our environment, we release a set of benchmark tasks focusing on evaluating the functional correctness of task completions. The tasks in our benchmark are diverse, long-horizon, and designed to emulate tasks that humans routinely perform on the internet. We experiment with several baseline agents, integrating recent techniques such as reasoning before acting. The results demonstrate that solving complex tasks is challenging: our best GPT-4-based agent only achieves an end-to-end task success rate of 14.41%, significantly lower than the human performance of 78.24%. These results highlight the need for further development of robust agents, that current state-of-the-art large language models are far from perfect performance in these real-life tasks, and that WebArena can be used to measure such progress.

Key facts. Cited by 1,908 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 42 of its 66 references that are in the map. First posted on arXiv 25 Jul 2023.

Builds on

42 of 66 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun
  2. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul

Other references · 40

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  10. 2016OpenAI Gymcited by 1,684
  11. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  12. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
28 more
  1. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  2. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  3. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  4. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  5. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  6. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  7. 2022PAL: Program-aided Language Modelscited by 635
  8. 2023Active Retrieval Augmented Generationcited by 506
  9. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  10. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  11. 2023Language Models can Solve Computer Taskscited by 371
  12. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
  13. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  14. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
  15. 2023Large Language Model Guided Tree-of-Thoughtcited by 218
  16. 2020The NetHack Learning Environmentcited by 196
  17. 2020Mapping Natural Language Instructions to Mobile UI Action Sequencescited by 180
  18. 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
  19. 2017IQA: Visual Question Answering in Interactive Environmentscited by 154
  20. 2018Transforming Question Answering Datasets Into Natural Language Inference Datasetscited by 99
  21. 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
  22. 2022Execution-Based Evaluation for Open-Domain Code Generationcited by 89
  23. 2023From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfacescited by 67
  24. 2021Grounding Open-Domain Instructions to Automate Web Support Taskscited by 43
  25. 2023Hierarchical Prompting Assists Large Language Model on Web Navigationcited by 36
  26. 2022Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Datacited by 22
  27. 2023Take a Break in the Middle: Investigating Subgoals towards Hierarchical Script Generationcited by 7
  28. 2019CraftAssist Instruction Parsing: Semantic Parsing for a Minecraft Assistantcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 44; 2024: 225; 2025: 477; 2026: 1162.

1,908 papers.

Sort:

By area:

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  3. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  6. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  7. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  8. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  9. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  10. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  11. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  12. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
38 more
  1. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
  2. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  3. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  4. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  5. 2023Cognitive Architectures for Language Agentscited by 422
  6. 2024Understanding the planning of LLM agents: A surveycited by 414
  7. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
  8. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  9. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
  10. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
  11. 2024Agent Workflow Memorycited by 342
  12. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
  13. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  14. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  15. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
  16. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  17. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  18. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306
  19. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  20. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  21. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
  22. 2025Memory in the Age of AI Agentscited by 285
  23. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 272
  24. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
  25. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
  26. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  27. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  28. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  29. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 253
  30. 2023FireAct: Toward Language Agent Fine-tuningcited by 247
  31. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  32. 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agentscited by 236
  33. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
  34. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 226
  35. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 223
  36. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
  37. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  38. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212

50 most cited of 1,908 All 1,908 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026WebFactory: Automated Compression of Foundational Language Intelligence into Grounded Web Agents
  2. 2026Training Needs Trustworthy Worlds: Verified Synthetic Web Environments for Agent Learning
  3. 2026InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training
  4. 2026OpAgent: Operator Agent for Web Navigation
  5. 2025Magentic-UI: Towards Human-in-the-loop Agentic Systems
  6. 2025BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions
  7. 2024Unveiling Disparities in Web Task Handling Between Human and Web Agent
  8. 2026AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments