# Empirical Evaluation of Mutation-based Test Prioritization Techniques

arXiv: https://arxiv.org/abs/1709.04631 · HTML: https://mlsanity.com/paper/1709.04631/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2017.

## Built on by (most cited first, 4 of 4)

- [Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling](https://mlsanity.com/paper/2507.23370/) (2025)
- [Mutation-based Fault Localization of Deep Neural Networks](https://mlsanity.com/paper/2309.05067/) (2023)
- [CertPri: Certifiable Prioritization for Deep Neural Networks via Movement Cost in Feature Space](https://mlsanity.com/paper/2307.09375/) (2023)
- [Metamorphic Relation Prioritization for Effective Regression Testing](https://mlsanity.com/paper/2109.09798/) (2021)

## Abstract (arXiv)

We propose a new test case prioritization technique that combines both mutation-based and diversity-based approaches. Our diversity-aware mutation-based technique relies on the notion of mutant distinguishment, which aims to distinguish one mutant's behavior from another, rather than from the original program. We empirically investigate the relative cost and effectiveness of the mutation-based prioritization techniques (i.e., using both the traditional mutant kill and the proposed mutant distinguishment) with 352 real faults and 553,477 developer-written test cases. The empirical evaluation considers both the traditional and the diversity-aware mutation criteria in various settings: single-objective greedy, hybrid, and multi-objective optimization. The results show that there is no single dominant technique across all the studied faults. To this end, revwe we show when and the reason why each one of the mutation-based prioritization criteria performs poorly, using a graphical model called Mutant Distinguishment Graph (MDG) that demonstrates the distribution of the fault detecting test cases with respect to mutant kills and distinguishment.