# Systematic Literature Review on Application of Machine Learning in Continuous Integration

arXiv: https://arxiv.org/abs/2305.12695 · HTML: https://mlsanity.com/paper/2305.12695/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 100 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 13 of 13)

- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice](https://mlsanity.com/paper/2007.15745/) (2020)
- [Continuous Integration, Delivery and Deployment: A Systematic Review on Approaches, Tools, Challenges and Practices](https://mlsanity.com/paper/1703.07019/) (2017)
- [Detection of data drift and outliers affecting machine learning model performance over time](https://mlsanity.com/paper/2012.09258/) (2020)
- [Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration](https://mlsanity.com/paper/1811.04122/) (2018)
- [Automatically detecting data drift in machine learning classifiers](https://mlsanity.com/paper/2111.05672/) (2021)
- [Reinforcement Learning for Test Case Prioritization](https://mlsanity.com/paper/2011.01834/) (2020)
- [Test Case Selection and Prioritization Using Machine Learning: A Systematic Literature Review](https://mlsanity.com/paper/2106.13891/) (2021)
- [DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing](https://mlsanity.com/paper/2110.07443/) (2021)
- [Scalable and Accurate Test Case Prioritization in Continuous Integration Contexts](https://mlsanity.com/paper/2109.13168/) (2021)
- [Data stream mining for predicting software build outcomes using source code metrics](https://mlsanity.com/paper/1604.05503/) (2016)
- [Test Prioritization in Continuous Integration Environments](https://mlsanity.com/paper/1809.00143/) (2018)
- [Machine Learning for Software Engineering: A Systematic Mapping](https://mlsanity.com/paper/2005.13299/) (2020)

## Built on by (most cited first, 2 of 2)

- [Are Latent Vulnerabilities Hidden Gems for Software Vulnerability Prediction? An Empirical Study](https://mlsanity.com/paper/2401.11105/) (2024)
- [Software Vulnerability Prediction in Low-Resource Languages: An Empirical Study of CodeBERT and ChatGPT](https://mlsanity.com/paper/2404.17110/) (2024)

## Abstract (arXiv)

This research conducted a systematic review of the literature on machine learning (ML)-based methods in the context of Continuous Integration (CI) over the past 22 years. The study aimed to identify and describe the techniques used in ML-based solutions for CI and analyzed various aspects such as data engineering, feature engineering, hyper-parameter tuning, ML models, evaluation methods, and metrics. In this paper, we have depicted the phases of CI testing, the connection between them, and the employed techniques in training the ML method phases. We presented nine types of data sources and four taken steps in the selected studies for preparing the data. Also, we identified four feature types and nine subsets of data features through thematic analysis of the selected studies. Besides, five methods for selecting and tuning the hyper-parameters are shown. In addition, we summarised the evaluation methods used in the literature and identified fifteen different metrics. The most commonly used evaluation methods were found to be precision, recall, and F1-score, and we have also identified five methods for evaluating the performance of trained ML models. Finally, we have presented the relationship between ML model types, performance measurements, and CI phases. The study provides valuable insights for researchers and practitioners interested in ML-based methods in CI and emphasizes the need for further research in this area.