# Automatic Web Testing using Curiosity-Driven Reinforcement Learning

arXiv: https://arxiv.org/abs/2103.06018 · HTML: https://mlsanity.com/paper/2103.06018/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2021.

## Built on by (most cited first, 19 of 19)

- [HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation](https://mlsanity.com/paper/2109.05490/) (2021)
- [BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasets](https://mlsanity.com/paper/2210.04688/) (2022)
- [Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems](https://mlsanity.com/paper/2210.15432/) (2022)
- [GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesis](https://mlsanity.com/paper/2205.13728/) (2022)
- [Autonomous Large Language Model Agents Enabling Intent-Driven Mobile GUI Testing](https://mlsanity.com/paper/2311.08649/) (2023)
- [WebWISE: Web Interface Control and Sequential Exploration with Large Language Models](https://mlsanity.com/paper/2310.16042/) (2023)
- [NaviQAte: Functionality-Guided Web Application Navigation](https://mlsanity.com/paper/2409.10741/) (2024)
- [A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs](https://mlsanity.com/paper/2411.07098/) (2024)
- [Neural Episodic Control with State Abstraction](https://mlsanity.com/paper/2301.11490/) (2023)
- [PAnDR: Fast Adaptation to New Environments from Offline Experiences via Decoupling Policy and Environment Representations](https://mlsanity.com/paper/2204.02877/) (2022)
- [Constrained Adversarial Learning for Automated Software Testing: a literature review](https://mlsanity.com/paper/2303.07546/) (2023)
- [AI Agents for Web Testing: A Case Study in the Wild](https://mlsanity.com/paper/2509.05197/) (2025)
- [WebTestPilot: Agentic End-to-End Web Testing against Natural Language Specification by Inferring Oracles with Symbolized GUI Elements](https://mlsanity.com/paper/2602.11724/) (2026)
- [FuzzTheREST: An Intelligent Automated Black-box RESTful API Fuzzer](https://mlsanity.com/paper/2407.14361/) (2024)
- [LaF: Labeling-Free Model Selection for Automated Deep Neural Network Reusing](https://mlsanity.com/paper/2204.03994/) (2022)
- [The Integration of Machine Learning into Automated Test Generation: A Systematic Mapping Study](https://mlsanity.com/paper/2206.10210/) (2022)
- [AURORA: Navigating UI Tarpits via Automated Neural Screen Understanding](https://mlsanity.com/paper/2404.01240/) (2024)
- [Clutch Control: An Attention-based Combinatorial Bandit for Efficient Mutation in JavaScript Engine Fuzzing](https://mlsanity.com/paper/2510.12732/) (2025)
- [WebCraftBench: Evaluating Web Application Generation from a Software Testing Perspective](https://mlsanity.com/paper/2609.15387/) (2026)

## Abstract (arXiv)

Web testing has long been recognized as a notoriously difficult task. Even nowadays, web testing still heavily relies on manual efforts while automated web testing is far from achieving human-level performance. Key challenges in web testing include dynamic content update and deep bugs hiding under complicated user interactions and specific input values, which can only be triggered by certain action sequences in the huge search space. In this paper, we propose WebExplor, an automatic end-to-end web testing framework, to achieve an adaptive exploration of web applications. WebExplor adopts curiosity-driven reinforcement learning to generate high-quality action sequences (test cases) satisfying temporal logical relations. Besides, WebExplor incrementally builds an automaton during the online testing process, which provides high-level guidance to further improve the testing efficiency. We have conducted comprehensive evaluations of WebExplor on six real-world projects, a commercial SaaS web application, and performed an in-the-wild study of the top 50 web applications in the world. The results demonstrate that in most cases WebExplor can achieve a significantly higher failure detection rate, code coverage, and efficiency than existing state-of-the-art web testing techniques. WebExplor also detected 12 previously unknown failures in the commercial web application, which have been confirmed and fixed by the developers. Furthermore, our in-the-wild study further uncovered 3,466 exceptions and errors.