# WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Tasks

arXiv: https://arxiv.org/abs/2506.01952 · HTML: https://mlsanity.com/paper/2506.01952/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 22 of 22)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models](https://mlsanity.com/paper/2401.13919/) (2024)
- [Agent Workflow Memory](https://mlsanity.com/paper/2409.07429/) (2024)
- [WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?](https://mlsanity.com/paper/2403.07718/) (2024)
- [TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks](https://mlsanity.com/paper/2412.14161/) (2024)
- [WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning](https://mlsanity.com/paper/2411.02337/) (2024)
- [WebWalker: Benchmarking LLMs in Web Traversal](https://mlsanity.com/paper/2501.07572/) (2025)
- [An Illusion of Progress? Assessing the Current State of Web Agents](https://mlsanity.com/paper/2504.01382/) (2025)
- [The BrowserGym Ecosystem for Web Agent Research](https://mlsanity.com/paper/2412.05467/) (2024)
- [AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?](https://mlsanity.com/paper/2407.15711/) (2024)
- [Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation](https://mlsanity.com/paper/2410.13232/) (2024)
- [WebCanvas: Benchmarking Web Agents in Online Environments](https://mlsanity.com/paper/2406.12373/) (2024)
- [AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents](https://mlsanity.com/paper/2410.13825/) (2024)
- [WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks](https://mlsanity.com/paper/2407.05291/) (2024)
- [Beyond Browsing: API-Based Web Agents](https://mlsanity.com/paper/2410.16464/) (2024)
- [MMInA: Benchmarking Multihop Multimodal Internet Agents](https://mlsanity.com/paper/2404.09992/) (2024)
- [Towards Enterprise-Ready Computer Using Generalist Agent](https://mlsanity.com/paper/2503.01861/) (2025)
- [Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMs](https://mlsanity.com/paper/2502.07942/) (2025)

## Built on by (most cited first, 30 of 30)

- [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025)
- [ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory](https://mlsanity.com/paper/2509.25140/) (2025)
- [ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents](https://mlsanity.com/paper/2508.14040/) (2025)
- [ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation](https://mlsanity.com/paper/2507.04952/) (2025)
- [OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks](https://mlsanity.com/paper/2606.29537/) (2026)
- [Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges](https://mlsanity.com/paper/2510.23883/) (2025)
- [Graph-based Agent Memory: Taxonomy, Techniques, and Applications](https://mlsanity.com/paper/2602.05665/) (2026)
- [From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms](https://mlsanity.com/paper/2605.06716/) (2026)
- [WebMall -- A Multi-Shop Benchmark for Evaluating Web Agents](https://mlsanity.com/paper/2508.13024/) (2025)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks](https://mlsanity.com/paper/2510.02418/) (2025)
- [AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agents](https://mlsanity.com/paper/2512.23343/) (2025)
- [UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experience](https://mlsanity.com/paper/2603.24533/) (2026)
- [How Well Does Agent Development Reflect Real-World Work?](https://mlsanity.com/paper/2603.01203/) (2026)
- [AgentBay: A Hybrid Interaction Sandbox for Seamless Human-AI Intervention in Agentic Systems](https://mlsanity.com/paper/2512.04367/) (2025)
- [WebFactory: Automated Compression of Foundational Language Intelligence into Grounded Web Agents](https://mlsanity.com/paper/2603.05044/) (2026)
- [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026)
- [Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents](https://mlsanity.com/paper/2606.15017/) (2026)
- [WebDART: Dynamic Decomposition and Re-planning for Complex Web Tasks](https://mlsanity.com/paper/2510.06587/) (2025)
- [WebForge: Breaking the Realism-Reproducibility-Scalability Trilemma in Browser Agent Benchmark](https://mlsanity.com/paper/2604.10988/) (2026)
- [WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing](https://mlsanity.com/paper/2603.25226/) (2026)
- [OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents](https://mlsanity.com/paper/2608.05013/) (2026)
- [WebGen-V Bench: Structured Representation for Enhancing Visual Design in LLM-based Web Generation and Evaluation](https://mlsanity.com/paper/2510.15306/) (2025)
- [GUI Agents with Reinforcement Learning: Toward Digital Inhabitants](https://mlsanity.com/paper/2604.27955/) (2026)
- [WebChallenger: A Reliable and Efficient Generalist Web Agent](https://mlsanity.com/paper/2606.10423/) (2026)
- [360CityArena: A Realistic Virtual Urban Navigation Benchmark for Embodied Agents](https://mlsanity.com/paper/2608.08814/) (2026)
- [MARCA: A Checklist-Based Benchmark for Multilingual Web Search](https://mlsanity.com/paper/2604.14448/) (2026)
- [On the Internet, Nobody Knows You're an LLM Bot: Unmasking Web Agents with Multi-Layer Fingerprinting](https://mlsanity.com/paper/2606.30119/) (2026)
- [Falsifiable Commitment Planning for Self-Correcting Web Agents](https://mlsanity.com/paper/2607.24167/) (2026)
- [EnterpriseBench: Benchmarking LLM Agents on Enterprise-Level Strategic Reasoning and Decision-Making](https://mlsanity.com/paper/2609.37658/) (2026)

## Abstract (arXiv)

Powered by large language models (LLMs), web browsing agents operate graphical user interfaces in a human-like manner, offering a transparent and general framework for automating web-based tasks. As these agents rapidly improve and achieve strong performance on existing benchmarks such as WebArena, a key question arises: textitCan current benchmarks still accurately evaluate the capabilities of increasingly powerful agents, especially for more tedious and cognitively demanding tasks? In this paper, we present WebChoreArena, a substantial extension of WebArena designed to push beyond general browsing scenarios. WebChoreArena introduces 532 carefully curated tasks developed over 300+ hours, explicitly targeting more labor-intensive and complex web chores. It systematically expands the evaluation space along three critical dimensions: (i) Massive Memory, requiring agents to accurately retain and retrieve large amounts of information from observations; (ii) Calculation, demanding precise mathematical reasoning over collected information; and (iii) Long-Term Memory, necessitating consistent information tracking across multiple webpages. Built directly on top of the four reproducible WebArena environments, WebChoreArena ensures strict compatibility and enables fair, controlled comparisons with prior work. Our experimental results demonstrate that as LLMs evolve, significant performance improvements are observed on WebChoreArena. These findings suggest that WebChoreArena is well-suited to measure the advancement of state-of-the-art LLMs with greater clarity. Nevertheless, the results also indicate that even with GPT-5, there remains substantial room for improvement compared to WebArena, highlighting the increased challenges posed by WebChoreArena.