# The Integration of Machine Learning into Automated Test Generation: A Systematic Mapping Study

arXiv: https://arxiv.org/abs/2206.10210 · HTML: https://mlsanity.com/paper/2206.10210/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 143 references that are in the map.
- First posted on arXiv 21 Jun 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [TOGA: A Neural Method for Test Oracle Generation](https://mlsanity.com/paper/2109.09262/) (2021)
- [Automated Video Game Testing Using Synthetic and Human-Like Agents](https://mlsanity.com/paper/1906.00317/) (2019)
- [Automatic Web Testing using Curiosity-Driven Reinforcement Learning](https://mlsanity.com/paper/2103.06018/) (2021)
- [Neural Network Guided Evolutionary Fuzzing for Finding Traffic Violations of Autonomous Vehicles](https://mlsanity.com/paper/2109.06126/) (2021)
- [GLIB: Towards Automated Test Oracle for Graphically-Rich Applications](https://mlsanity.com/paper/2106.10507/) (2021)
- [Adaptive Metamorphic Testing with Contextual Bandits](https://mlsanity.com/paper/1910.00262/) (2019)
- [Using Machine Learning to Generate Test Oracles: A Systematic Literature Review](https://mlsanity.com/paper/2107.00906/) (2021)
- [Learning to Encode and Classify Test Executions](https://mlsanity.com/paper/2001.02444/) (2020)
- [Intelligent Agent-Based Stimulation for Testing Robotic Software in Human-Robot Interactions](https://mlsanity.com/paper/1604.05508/) (2016)
- [Uncertainty-Driven Black-Box Test Data Generation](https://mlsanity.com/paper/1608.03181/) (2016)
- [Learning How to Search: Generating Effective Test Cases Through Adaptive Fitness Function Selection](https://mlsanity.com/paper/2102.04822/) (2021)
- [On Introducing Automatic Test Case Generation in Practice: A Success Story and Lessons Learned](https://mlsanity.com/paper/2103.00465/) (2021)
- [Automated Performance Testing Based on Active Deep Learning](https://mlsanity.com/paper/2104.02102/) (2021)
- [An Autonomous Performance Testing Framework using Self-Adaptive Fuzzy Reinforcement Learning](https://mlsanity.com/paper/1908.06900/) (2019)
- [Using Semi-Supervised Learning for Predicting Metamorphic Relations](https://mlsanity.com/paper/1802.07324/) (2018)
- [Towards Automated Metamorphic Test Identification for Ocean System Models](https://mlsanity.com/paper/2103.09782/) (2021)
- [Automated identification of metamorphic test scenarios for an ocean-modeling application](https://mlsanity.com/paper/2009.01554/) (2020)

## Built on by (most cited first, 1 of 1)

- [Review of Machine Learning for Micro-Electronic Design Verification](https://mlsanity.com/paper/2503.11687/) (2025)

## Abstract (arXiv)

Context: Machine learning (ML) may enable effective automated test generation. Objective: We characterize emerging research, examining testing practices, researcher goals, ML techniques applied, evaluation, and challenges. Methods: We perform a systematic mapping on a sample of 124 publications. Results: ML generates input for system, GUI, unit, performance, and combinatorial testing or improves the performance of existing generation methods. ML is also used to generate test verdicts, property-based, and expected output oracles. Supervised learning - often based on neural networks - and reinforcement learning - often based on Q-learning - are common, and some publications also employ unsupervised or semi-supervised learning. (Semi-/Un-)Supervised approaches are evaluated using both traditional testing metrics and ML-related metrics (e.g., accuracy), while reinforcement learning is often evaluated using testing metrics tied to the reward function. Conclusion: Work-to-date shows great promise, but there are open challenges regarding training data, retraining, scalability, evaluation complexity, ML algorithms employed - and how they are applied - benchmarks, and replicability. Our findings can serve as a roadmap and inspiration for researchers in this field.