# The Impact of Class Rebalancing Techniques on the Performance and Interpretation of Defect Prediction Models

arXiv: https://arxiv.org/abs/1801.10269 · HTML: https://mlsanity.com/paper/1801.10269/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jan 2018.

## Built on by (most cited first, 29 of 29)

- [The impact of feature importance methods on the interpretation of defect classifiers](https://mlsanity.com/paper/2202.02389/) (2022)
- [DeepCVA: Automated Commit-level Vulnerability Assessment with Deep Multi-task Learning](https://mlsanity.com/paper/2108.08041/) (2021)
- [Towards a consistent interpretation of AIOps models](https://mlsanity.com/paper/2202.02298/) (2022)
- [On the Use of Fine-grained Vulnerable Code Statements for Software Vulnerability Assessment Models](https://mlsanity.com/paper/2203.08417/) (2022)
- [Learning Defect Prediction from Unrealistic Data](https://mlsanity.com/paper/2311.00931/) (2023)
- [SQAPlanner: Generating Data-Informed Software Quality Improvement Plans](https://mlsanity.com/paper/2102.09687/) (2021)
- [Past, Present, and Future of Bug Tracking in the Generative AI Era](https://mlsanity.com/paper/2510.08005/) (2025)
- [Efficient Fraud Detection Using Deep Boosting Decision Trees](https://mlsanity.com/paper/2302.05918/) (2023)
- [Graph-Based Machine Learning Improves Just-in-Time Defect Prediction](https://mlsanity.com/paper/2110.05371/) (2021)
- [Bridging Expert Knowledge with Deep Learning Techniques for Just-In-Time Defect Prediction](https://mlsanity.com/paper/2403.11079/) (2024)
- [Causal Inference Tools for a Better Evaluation of Machine Learning](https://mlsanity.com/paper/2410.01392/) (2024)
- [Fairer Analysis and Demographically Balanced Face Generation for Fairer Face Verification](https://mlsanity.com/paper/2412.03349/) (2024)
- [Assessing the Early Bird Heuristic (for Predicting Project Quality)](https://mlsanity.com/paper/2105.11082/) (2021)
- [SINGA-Easy: An Easy-to-Use Framework for MultiModal Analysis](https://mlsanity.com/paper/2108.02572/) (2021)
- [VisualTextRank: Unsupervised Graph-based Content Extraction for Automating Ad Text to Image Search](https://mlsanity.com/paper/2108.02725/) (2021)
- [Text is NOT Enough: Integrating Visual Impressions into Open-domain Dialogue Generation](https://mlsanity.com/paper/2109.05778/) (2021)
- [Discovering Personalized Semantics for Soft Attributes in Recommender Systems using Concept Activation Vectors](https://mlsanity.com/paper/2202.02830/) (2022)
- [On The Effectiveness of One-Class Support Vector Machine in Different Defect Prediction Scenarios](https://mlsanity.com/paper/2202.12074/) (2022)
- [Reinforced MOOCs Concept Recommendation in Heterogeneous Information Networks](https://mlsanity.com/paper/2203.11011/) (2022)
- [Explanation-Guided Fairness Testing through Genetic Algorithm](https://mlsanity.com/paper/2205.08335/) (2022)
- [Using Graph Neural Networks for Program Termination](https://mlsanity.com/paper/2207.14648/) (2022)
- [Tribe or Not? Critical Inspection of Group Differences Using TribalGram](https://mlsanity.com/paper/2303.09664/) (2023)
- [Detecting Concept Drift for the reliability prediction of Software Defects using Instance Interpretation](https://mlsanity.com/paper/2305.16323/) (2023)
- [Mining Action Rules for Defect Reduction Planning](https://mlsanity.com/paper/2405.13740/) (2024)
- [rECGnition_v1.0: Arrhythmia detection using cardiologist-inspired multi-modal architecture incorporating demographic attributes in ECG](https://mlsanity.com/paper/2410.18985/) (2024)
- [FairPO: Robust Preference Optimization for Fair Multi-Label Learning](https://mlsanity.com/paper/2505.02433/) (2025)
- [Focus on What Matters: Fisher-Guided Adaptive Multimodal Fusion for Vulnerability Detection](https://mlsanity.com/paper/2601.02438/) (2026)
- [Training Dynamics of Neural Software Defect Predictors under Coupled Data-Quality Issues](https://mlsanity.com/paper/2606.24968/) (2026)
- [From Backlog Items to Security Guidance: Towards Continuous Security Compliance](https://mlsanity.com/paper/2607.27374/) (2026)

## Abstract (arXiv)

Defect prediction models that are trained on class imbalanced datasets (i.e., the proportion of defective and clean modules is not equally represented) are highly susceptible to produce inaccurate prediction models. Prior research compares the impact of class rebalancing techniques on the performance of defect prediction models. Prior research efforts arrive at contradictory conclusions due to the use of different choice of datasets, classification techniques, and performance measures. Such contradictory conclusions make it hard to derive practical guidelines for whether class rebalancing techniques should be applied in the context of defect prediction models. In this paper, we investigate the impact of 4 popularly-used class rebalancing techniques on 10 commonly-used performance measures and the interpretation of defect prediction models. We also construct statistical models to better understand in which experimental design settings that class rebalancing techniques are beneficial for defect prediction models. Through a case study of 101 datasets that span across proprietary and open-source systems, we recommend that class rebalancing techniques are necessary when quality assurance teams wish to increase the completeness of identifying software defects (i.e., Recall). However, class rebalancing techniques should be avoided when interpreting defect prediction models. We also find that class rebalancing techniques do not impact the AUC measure. Hence, AUC should be used as a standard measure when comparing defect prediction models.