# Practitioners' Perceptions of the Goals and Visual Explanations of Defect Prediction Models

arXiv: https://arxiv.org/abs/2102.12007 · HTML: https://mlsanity.com/paper/2102.12007/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2021.

## Built on by (most cited first, 9 of 9)

- [Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey](https://mlsanity.com/paper/2310.17903/) (2023)
- [Are Latent Vulnerabilities Hidden Gems for Software Vulnerability Prediction? An Empirical Study](https://mlsanity.com/paper/2401.11105/) (2024)
- [Ethics in the Age of AI: An Analysis of AI Practitioners' Awareness and Challenges](https://mlsanity.com/paper/2307.10057/) (2023)
- [Looking into Black Box Code Language Models](https://mlsanity.com/paper/2407.04868/) (2024)
- [Neuron-Guided Interpretation of Code LLMs: Where, Why, and How?](https://mlsanity.com/paper/2512.19980/) (2025)
- [Studying the explanations for the automated prediction of bug and non-bug issues using LIME and SHAP](https://mlsanity.com/paper/2209.07623/) (2022)
- [Predicting Line-Level Defects by Capturing Code Contexts with Hierarchical Transformers](https://mlsanity.com/paper/2312.11889/) (2023)
- [Practitioners' Challenges and Perceptions of CI Build Failure Predictions at Atlassian](https://mlsanity.com/paper/2402.09651/) (2024)
- [Towards Explainable Test Case Prioritisation with Learning-to-Rank Models](https://mlsanity.com/paper/2405.13786/) (2024)

## Abstract (arXiv)

Software defect prediction models are classifiers that are constructed from historical software data. Such software defect prediction models have been proposed to help developers optimize the limited Software Quality Assurance (SQA) resources and help managers develop SQA plans. Prior studies have different goals for their defect prediction models and use different techniques for generating visual explanations of their models. Yet, it is unclear what are the practitioners' perceptions of (1) these defect prediction model goals, and (2) the model-agnostic techniques used to visualize these models. We conducted a qualitative survey to investigate practitioners' perceptions of the goals of defect prediction models and the model-agnostic techniques used to generate visual explanations of defect prediction models. We found that (1) 82%-84% of the respondents perceived that the three goals of defect prediction models are useful; (2) LIME is the most preferred technique for understanding the most important characteristics that contributed to a prediction of a file, while ANOVA/VarImp is the second most preferred technique for understanding the characteristics that are associated with software defects in the past. Our findings highlight the significance of investigating how to improve the understanding of defect prediction models and their predictions. Hence, model-agnostic techniques from explainable AI domain may help practitioners to understand defect prediction models and their predictions.