# Predictive Test Selection

arXiv: https://arxiv.org/abs/1810.05286 · HTML: https://mlsanity.com/paper/1810.05286/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2018.

## Built on by (most cited first, 9 of 9)

- [DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing](https://mlsanity.com/paper/2110.07443/) (2021)
- [Comparative Study of Machine Learning Test Case Prioritization for Continuous Integration Testing](https://mlsanity.com/paper/2204.10899/) (2022)
- [AI for Agile development: a Meta-Analysis](https://mlsanity.com/paper/2305.08093/) (2023)
- [Targeted Test Selection Approach in Continuous Integration](https://mlsanity.com/paper/2509.10279/) (2025)
- [An Empirical Study of Sample Selection Strategies for Large Language Model Repair](https://mlsanity.com/paper/2510.20428/) (2025)
- [Regression Test Selection for Updated Capability Modules in Compositional ML Systems via Atomic-Quality Probes](https://mlsanity.com/paper/2604.26689/) (2026)
- [Efficient Benchmarking in Production: A Study of an Evolving LLM Agent](https://mlsanity.com/paper/2609.21267/) (2026)
- [FastCI: Efficient GPU-Intensive CI for LLM Training Frameworks](https://mlsanity.com/paper/2610.01967/) (2026)
- [Software World Models: From Consequence Prediction to Decision Value](https://mlsanity.com/paper/2610.04940/) (2026)

## Abstract (arXiv)

Change-based testing is a key component of continuous integration at Facebook. However, a large number of tests coupled with a high rate of changes committed to our monolithic repository make it infeasible to run all potentially-impacted tests on each change. We propose a new predictive test selection strategy which selects a subset of tests to exercise for each change submitted to the continuous integration system. The strategy is learned from a large dataset of historical test outcomes using basic machine learning techniques. Deployed in production, the strategy reduces the total infrastructure cost of testing code changes by a factor of two, while guaranteeing that over 95% of individual test failures and over 99.9% of faulty changes are still reported back to developers. The method we present here also accounts for the non-determinism of test outcomes, also known as test flakiness.