# DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing

arXiv: https://arxiv.org/abs/2110.07443 · HTML: https://mlsanity.com/paper/2110.07443/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 38 references that are in the map.
- First posted on arXiv 14 Oct 2021.

## Builds on (most cited first, 6 of 6)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Mish: A Self Regularized Non-Monotonic Activation Function](https://mlsanity.com/paper/1908.08681/) (2019)
- [Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration](https://mlsanity.com/paper/1811.04122/) (2018)
- [UBL: an R package for Utility-based Learning](https://mlsanity.com/paper/1604.08079/) (2016)
- [Predictive Test Selection](https://mlsanity.com/paper/1810.05286/) (2018)

## Built on by (most cited first, 8 of 8)

- [Systematic Literature Review on Application of Learning-based Approaches in Continuous Integration](https://mlsanity.com/paper/2406.19765/) (2024)
- [SoK: Machine Learning for Continuous Integration](https://mlsanity.com/paper/2304.02829/) (2023)
- [Systematic Literature Review on Application of Machine Learning in Continuous Integration](https://mlsanity.com/paper/2305.12695/) (2023)
- [From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap](https://mlsanity.com/paper/2410.20791/) (2024)
- [Comparative Study of Machine Learning Test Case Prioritization for Continuous Integration Testing](https://mlsanity.com/paper/2204.10899/) (2022)
- [Industry-academia research collaboration and knowledge co-creation: Patterns and anti-patterns](https://mlsanity.com/paper/2204.14180/) (2022)
- [Targeted Test Selection Approach in Continuous Integration](https://mlsanity.com/paper/2509.10279/) (2025)
- [The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)](https://mlsanity.com/paper/2505.10640/) (2025)

## Abstract (arXiv)

Continuous integration testing is an important step in the modern software engineering life cycle. Test prioritization is a method that can improve the efficiency of continuous integration testing by selecting test cases that can detect faults in the early stage of each cycle. As continuous integration testing produces voluminous test execution data, test history is a commonly used artifact in test prioritization. However, existing test prioritization techniques for continuous integration either cannot handle large test history or are optimized for using a limited number of historical test cycles. We show that such a limitation can decrease fault detection effectiveness of prioritized test suites. This work introduces DeepOrder, a deep learning-based model that works on the basis of regression machine learning. DeepOrder ranks test cases based on the historical record of test executions from any number of previous test cycles. DeepOrder learns failed test cases based on multiple factors including the duration and execution status of test cases. We experimentally show that deep neural networks, as a simple regression model, can be efficiently used for test case prioritization in continuous integration testing. DeepOrder is evaluated with respect to time-effectiveness and fault detection effectiveness in comparison with an industry practice and the state of the art approaches. The results show that DeepOrder outperforms the industry practice and state-of-the-art test prioritization approaches in terms of these two metrics.