# AI Agents for Web Testing: A Case Study in the Wild

arXiv: https://arxiv.org/abs/2509.05197 · HTML: https://mlsanity.com/paper/2509.05197/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 34 references that are in the map.
- First posted on arXiv 5 Sep 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 21 of 21)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V](https://mlsanity.com/paper/2310.11441/) (2023)
- [An Introduction to Vision-Language Modeling](https://mlsanity.com/paper/2405.17247/) (2024)
- [Tree Search for Language Model Agents](https://mlsanity.com/paper/2407.01476/) (2024)
- [ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning](https://mlsanity.com/paper/2410.02052/) (2024)
- [Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy Planning](https://mlsanity.com/paper/2305.13660/) (2023)
- [Explore, Select, Derive, and Recall: Augmenting LLM with Human-like Memory for Mobile Task Automation](https://mlsanity.com/paper/2312.03003/) (2023)
- [Make LLM a Testing Expert: Bringing Human-like Interaction to Mobile GUI Testing via Functionality-aware Decisions](https://mlsanity.com/paper/2310.15780/) (2023)
- [UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design](https://mlsanity.com/paper/2502.12561/) (2025)
- [Fill in the Blank: Context-aware Automated Text Input Generation for Mobile GUI Testing](https://mlsanity.com/paper/2212.04732/) (2022)
- [AutoDroid-V2: Boosting SLM-based GUI Agents via Code Generation](https://mlsanity.com/paper/2412.18116/) (2024)
- [Automatic Web Testing using Curiosity-Driven Reinforcement Learning](https://mlsanity.com/paper/2103.06018/) (2021)
- [Autonomous Large Language Model Agents Enabling Intent-Driven Mobile GUI Testing](https://mlsanity.com/paper/2311.08649/) (2023)
- [Testing the Limits: Unusual Text Inputs Generation for Mobile App Crash Detection with Large Language Model](https://mlsanity.com/paper/2310.15657/) (2023)
- [NaviQAte: Functionality-Guided Web Application Navigation](https://mlsanity.com/paper/2409.10741/) (2024)
- [ResTest: Targeted Coverage of Residual Not-Covered Code Using Large Language Models for Web GUI Testing](https://mlsanity.com/paper/2506.00520/) (2025)
- [Feature-Driven End-To-End Test Generation](https://mlsanity.com/paper/2408.01894/) (2024)
- [Leveraging Large Vision Language Model For Better Automatic Web GUI Testing](https://mlsanity.com/paper/2410.12157/) (2024)
- [Automated Web Application Testing: End-to-End Test Case Generation with Large Language Models and Screen Transition Graphs](https://mlsanity.com/paper/2506.02529/) (2025)
- [Neural Embeddings for Web Testing](https://mlsanity.com/paper/2306.07400/) (2023)

## Built on by (most cited first, 6 of 6)

- [WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing](https://mlsanity.com/paper/2603.25226/) (2026)
- [DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agents](https://mlsanity.com/paper/2605.17439/) (2026)
- [WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts](https://mlsanity.com/paper/2606.03220/) (2026)
- [PerceptUI: LLM Agents as Human-Aligned Synthetic Users for UI/UX Evaluation](https://mlsanity.com/paper/2606.05697/) (2026)
- [Framework and Benchmark for Code-Driven Agentic Testing in Web Development](https://mlsanity.com/paper/2609.00081/) (2026)
- [Automatic multimodal UX improvement recommendations from LLM agent user simulations](https://mlsanity.com/paper/2609.22971/) (2026)

## Abstract (arXiv)

Automated web testing plays a critical role in ensuring high-quality user experiences and delivering business value. Traditional approaches primarily focus on code coverage and load testing, but often fall short of capturing complex user behaviors, leaving many usability issues undetected. The emergence of large language models (LLM) and AI agents opens new possibilities for web testing by enabling human-like interaction with websites and a general awareness of common usability problems. In this work, we present WebProber, a prototype AI agent-based web testing framework. Given a URL, WebProber autonomously explores the website, simulating real user interactions, identifying bugs and usability issues, and producing a human-readable report. We evaluate WebProber through a case study of 120 academic personal websites, where it uncovered 29 usability issues--many of which were missed by traditional tools. Our findings highlight agent-based testing as a promising direction while outlining directions for developing next-generation, user-centered testing frameworks.