# An ensemble meta-estimator to predict source code testability

arXiv: https://arxiv.org/abs/2208.09614 · HTML: https://mlsanity.com/paper/2208.09614/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 82 references that are in the map.
- First posted on arXiv 20 Aug 2022.

## Builds on (most cited first, 3 of 3)

- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018)
- [A survey on software testability](https://mlsanity.com/paper/1801.02201/) (2018)

## Built on by (most cited first, 5 of 5)

- [A systematic literature review on source code similarity measurement and clone detection: techniques, applications, and challenges](https://mlsanity.com/paper/2306.16171/) (2023)
- [How Diversely Can Language Models Solve Problems? Exploring the Algorithmic Diversity of Model-Generated Code](https://mlsanity.com/paper/2503.00691/) (2025)
- [Is Functional Correctness Enough to Evaluate Code Language Models? Exploring Diversity of Generated Codes](https://mlsanity.com/paper/2408.14504/) (2024)
- [A systematic literature review on the code smells datasets and validation mechanisms](https://mlsanity.com/paper/2306.01377/) (2023)
- [Natural Language Requirements Testability Measurement Based on Requirement Smells](https://mlsanity.com/paper/2403.17479/) (2024)

## Abstract (arXiv)

Unlike most other software quality attributes, testability cannot be evaluated solely based on the characteristics of the source code. The effectiveness of the test suite and the budget assigned to the test highly impact the testability of the code under test. The size of a test suite determines the test effort and cost, while the coverage measure indicates the test effectiveness. Therefore, testability can be measured based on the coverage and number of test cases provided by a test suite, considering the test budget. This paper offers a new equation to estimate testability regarding the size and coverage of a given test suite. The equation has been used to label 23,000 classes belonging to 110 Java projects with their testability measure. The labeled classes were vectorized using 262 metrics. The labeled vectors were fed into a family of supervised machine learning algorithms, regression, to predict testability in terms of the source code metrics. Regression models predicted testability with an R2 of 0.68 and a mean squared error of 0.03, suitable in practice. Fifteen software metrics highly affecting testability prediction were identified using a feature importance analysis technique on the learned model. The proposed models have improved mean absolute error by 38% due to utilizing new criteria, metrics, and data compared with the relevant study on predicting branch coverage as a test criterion. As an application of testability prediction, it is demonstrated that automated refactoring of 42 smelly Java classes targeted at improving the 15 influential software metrics could elevate their testability by an average of 86.87%.