# WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing arXiv: https://arxiv.org/abs/2603.25226 ยท HTML: https://mlsanity.com/paper/2603.25226/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. - Builds on 29 of its 42 references that are in the map. - First posted on arXiv 26 Mar 2026. ## Placed in - LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/) ## Builds on (most cited first, 29 of 29) - [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023) - [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024) - [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026) - [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022) - [GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models](https://mlsanity.com/paper/2508.06471/) (2025) - [CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges](https://mlsanity.com/paper/2401.07339/) (2024) - [ShowUI: One Vision-Language-Action Model for GUI Visual Agent](https://mlsanity.com/paper/2411.17465/) (2024) - [UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2509.02544/) (2025) - [The BrowserGym Ecosystem for Web Agent Research](https://mlsanity.com/paper/2412.05467/) (2024) - [Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering](https://mlsanity.com/paper/2403.03163/) (2024) - [UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction](https://mlsanity.com/paper/2503.15661/) (2025) - [WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch](https://mlsanity.com/paper/2505.03733/) (2025) - [Qwen3-Coder-Next Technical Report](https://mlsanity.com/paper/2603.00729/) (2026) - [Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping](https://mlsanity.com/paper/2411.03292/) (2024) - [DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generation](https://mlsanity.com/paper/2506.06251/) (2025) - [Make LLM a Testing Expert: Bringing Human-like Interaction to Mobile GUI Testing via Functionality-aware Decisions](https://mlsanity.com/paper/2310.15780/) (2023) - [UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design](https://mlsanity.com/paper/2502.12561/) (2025) - [WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Tasks](https://mlsanity.com/paper/2506.01952/) (2025) - [UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback](https://mlsanity.com/paper/2406.07739/) (2024) - [WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics](https://mlsanity.com/paper/2601.02430/) (2026) - [WebUIBench: A Comprehensive Benchmark for Evaluating Multimodal Large Language Models in WebUI-to-Code](https://mlsanity.com/paper/2506.07818/) (2025) - [GUI Testing Arena: A Unified Benchmark for Advancing Autonomous GUI Testing Agent](https://mlsanity.com/paper/2412.18426/) (2024) - [Understanding Skills for OSS Communities on GitHub](https://mlsanity.com/paper/2209.02222/) (2022) - [Are Autonomous Web Agents Good Testers?](https://mlsanity.com/paper/2504.01495/) (2025) - [AI Agents for Web Testing: A Case Study in the Wild](https://mlsanity.com/paper/2509.05197/) (2025) - [GUITester: Enabling GUI Agents for Exploratory Defect Discovery](https://mlsanity.com/paper/2601.04500/) (2026) - [Erratum: Leveraging Flexible Tree Matching to Repair Broken Locators in Web Automation Scripts](https://mlsanity.com/paper/2106.04916/) (2021) - [WebTestPilot: Agentic End-to-End Web Testing against Natural Language Specification by Inferring Oracles with Symbolized GUI Elements](https://mlsanity.com/paper/2602.11724/) (2026) - [Automated Web Application Testing: End-to-End Test Case Generation with Large Language Models and Screen Transition Graphs](https://mlsanity.com/paper/2506.02529/) (2025) ## Built on by (most cited first, 4 of 4) - [Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generation](https://mlsanity.com/paper/2605.30000/) (2026) - [DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agents](https://mlsanity.com/paper/2605.17439/) (2026) - [WebGrader: Training LLMs for Web Development with Self-Evolving Programmatic Grader](https://mlsanity.com/paper/2608.06474/) (2026) - [Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability](https://mlsanity.com/paper/2605.07110/) (2026) ## Abstract (arXiv) The emergence of Large Language Models (LLMs) has catalyzed a paradigm shift in programming, giving rise to "vibe coding", where users can build complete projects and even control computers using natural language instructions. This paradigm has driven automated webpage development, but it introduces a new requirement about how to automatically verify whether the web functionalities are reliably implemented. Existing works struggle to adapt, relying on static visual similarity or predefined checklists that constrain their utility in open-ended environments. Furthermore, they overlook a vital aspect of software quality, namely latent logical constraints. To address these gaps, we introduce WebTestBench, a benchmark for evaluating end-to-end automated web testing. WebTestBench encompasses comprehensive dimensions across diverse web application categories. We decompose the testing process into two cascaded sub-tasks, checklist generation and defect detection, and propose WebTester, a baseline framework for this task. Evaluating popular LLMs with WebTester reveals severe challenges, including insufficient test completeness, detection bottlenecks, and long-horizon interaction unreliability. These findings expose a substantial gap between current computer-use agent capabilities and industrial-grade deployment demands. We hope that WebTestBench provides valuable insights and guidance for advancing end-to-end automated web testing. Our dataset and code are available at https://github.com/friedrichor/WebTestBench.