# Assessing and Improving the Mutation Testing Practice of PIT

arXiv: https://arxiv.org/abs/1601.02351 · HTML: https://mlsanity.com/paper/1601.02351/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jan 2016.

## Built on by (most cited first, 3 of 3)

- [Sentinel: A Hyper-Heuristic for the Generation of Mutant Reduction Strategies](https://mlsanity.com/paper/2103.07241/) (2021)
- [EduPluginBench: Executable Assurance for AI-Generated Educational Plugins](https://mlsanity.com/paper/2608.00739/) (2026)
- [The Quality of Claude AI-authored Python Tests Is Not Weaker Than Human-authored Tests](https://mlsanity.com/paper/2608.15188/) (2026)

## Abstract (arXiv)

Mutation testing is used extensively to support the experimentation of software engineering studies. Its application to real-world projects is possible thanks to modern tools that automate the whole mutation analysis process. However, popular mutation testing tools use a restrictive set of mutants which do not conform to the community standards as supported by the mutation testing literature. This can be problematic since the effectiveness of mutation depends on its mutants. We therefore examine how effective are the mutants of a popular mutation testing tool, named PIT, compared to comprehensive ones, as drawn from the literature and personal experience. We show that comprehensive mutants are harder to kill and encode faults not captured by the mutants of PIT for a range of 11% to 62% of the Java classes of the considered projects.