# A Comparison of Reinforcement Learning Frameworks for Software Testing Tasks

arXiv: https://arxiv.org/abs/2208.12136 · HTML: https://mlsanity.com/paper/2208.12136/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 60 references that are in the map.
- First posted on arXiv 25 Aug 2022.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [Noisy Networks for Exploration](https://mlsanity.com/paper/1706.10295/) (2017)
- [Continuous Deep Q-Learning with Model-based Acceleration](https://mlsanity.com/paper/1603.00748/) (2016)
- [Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/1712.06567/) (2017)
- [Dopamine: A Research Framework for Deep Reinforcement Learning](https://mlsanity.com/paper/1812.06110/) (2018)
- [Augmenting Automated Game Testing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.15819/) (2021)
- [Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration](https://mlsanity.com/paper/1811.04122/) (2018)
- [Deep Reinforcement Fuzzing](https://mlsanity.com/paper/1801.04589/) (2018)
- [Reinforcement Learning for Test Case Prioritization](https://mlsanity.com/paper/2011.01834/) (2020)
- [Using Reinforcement Learning for Load Testing of Video Games](https://mlsanity.com/paper/2201.06865/) (2022)
- [Deep Reinforcement Learning for Black-Box Testing of Android Apps](https://mlsanity.com/paper/2101.02636/) (2021)
- [LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations](https://mlsanity.com/paper/1808.07903/) (2018)
- [Learning Transferable Graph Exploration](https://mlsanity.com/paper/1910.12980/) (2019)
- [FuzzerGym: A Competitive Framework for Fuzzing and Learning](https://mlsanity.com/paper/1807.07490/) (2018)
- [Computer Games Are Serious Business and so is their Quality: Particularities of Software Testing in Game Development from the Perspective of Practitioners](https://mlsanity.com/paper/1812.05164/) (2018)
- [An Autonomous Performance Testing Framework using Self-Adaptive Fuzzy Reinforcement Learning](https://mlsanity.com/paper/1908.06900/) (2019)
- [Towards Efficient Detection and Optimal Response against Sophisticated Opponents](https://mlsanity.com/paper/1809.04240/) (2018)

## Built on by (most cited first, 3 of 3)

- [A Reference Architecture of Reinforcement Learning Frameworks](https://mlsanity.com/paper/2603.06413/) (2026)
- [On the Mistaken Assumption of Interchangeable Deep Reinforcement Learning Implementations](https://mlsanity.com/paper/2503.22575/) (2025)
- [CiRL: Open-Source Environments for Reinforcement Learning in Circular Economy and Net Zero](https://mlsanity.com/paper/2505.21536/) (2025)

## Abstract (arXiv)

Software testing activities scrutinize the artifacts and the behavior of a software product to find possible defects and ensure that the product meets its expected requirements. Recently, Deep Reinforcement Learning (DRL) has been successfully employed in complex testing tasks such as game testing, regression testing, and test case prioritization to automate the process and provide continuous adaptation. Practitioners can employ DRL by implementing from scratch a DRL algorithm or using a DRL framework. DRL frameworks offer well-maintained implemented state-of-the-art DRL algorithms to facilitate and speed up the development of DRL applications. Developers have widely used these frameworks to solve problems in various domains including software testing. However, to the best of our knowledge, there is no study that empirically evaluates the effectiveness and performance of implemented algorithms in DRL frameworks. Moreover, some guidelines are lacking from the literature that would help practitioners choose one DRL framework over another. In this paper, we empirically investigate the applications of carefully selected DRL algorithms on two important software testing tasks: test case prioritization in the context of Continuous Integration (CI) and game testing. For the game testing task, we conduct experiments on a simple game and use DRL algorithms to explore the game to detect bugs. Results show that some of the selected DRL frameworks such as Tensorforce outperform recent approaches in the literature. To prioritize test cases, we run experiments on a CI environment where DRL algorithms from different frameworks are used to rank the test cases. Our results show that the performance difference between implemented algorithms in some cases is considerable, motivating further investigation.