# Testing for Fault Diversity in Reinforcement Learning

arXiv: https://arxiv.org/abs/2403.15065 · HTML: https://mlsanity.com/paper/2403.15065/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2024.

## Built on by (most cited first, 3 of 3)

- [DiCriTest: Testing Scenario Generation for Decision-Making Agents Considering Diversity and Criticality](https://mlsanity.com/paper/2508.11514/) (2025)
- [Gimitest: A Comprehensive Tool for Testing Reinforcement Learning Policies](https://mlsanity.com/paper/2607.07029/) (2026)
- [DreamTest: World-Model Surrogates for Search-Based Testing of Deep Reinforcement Learning Agents](https://mlsanity.com/paper/2610.04494/) (2026)

## Abstract (arXiv)

Reinforcement Learning is the premier technique to approach sequential decision problems, including complex tasks such as driving cars and landing spacecraft. Among the software validation and verification practices, testing for functional fault detection is a convenient way to build trustworthiness in the learned decision model. While recent works seek to maximise the number of detected faults, none consider fault characterisation during the search for more diversity. We argue that policy testing should not find as many failures as possible (e.g., inputs that trigger similar car crashes) but rather aim at revealing as informative and diverse faults as possible in the model. In this paper, we explore the use of quality diversity optimisation to solve the problem of fault diversity in policy testing. Quality diversity (QD) optimisation is a type of evolutionary algorithm to solve hard combinatorial optimisation problems where high-quality diverse solutions are sought. We define and address the underlying challenges of adapting QD optimisation to the test of action policies. Furthermore, we compare classical QD optimisers to state-of-the-art frameworks dedicated to policy testing, both in terms of search efficiency and fault diversity. We show that QD optimisation, while being conceptually simple and generally applicable, finds effectively more diverse faults in the decision model, and conclude that QD-based policy testing is a promising approach.