# Is "Better Data" Better than "Better Data Miners"? (On the Benefits of Tuning SMOTE for Defect Prediction)

arXiv: https://arxiv.org/abs/1705.03697 · HTML: https://mlsanity.com/paper/1705.03697/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2017.

## Built on by (most cited first, 17 of 17)

- [FRUGAL: Unlocking SSL for Software Analytics](https://mlsanity.com/paper/2108.09847/) (2021)
- [DebtFree: Minimizing Labeling Cost in Self-Admitted Technical Debt Identification using Semi-Supervised Learning](https://mlsanity.com/paper/2201.10592/) (2022)
- [Learning from Very Little Data: On the Value of Landscape Analysis for Predicting Software Project Health](https://mlsanity.com/paper/2301.06577/) (2023)
- [Old but Gold: Reconsidering the value of feedforward learners for software analytics](https://mlsanity.com/paper/2101.06319/) (2021)
- [Dazzle: Using Optimized Generative Adversarial Networks to Address Security Data Class Imbalance Issue](https://mlsanity.com/paper/2203.11410/) (2022)
- [How to Find Actionable Static Analysis Warnings: A Case Study with FindBugs](https://mlsanity.com/paper/2205.10504/) (2022)
- [Assessing the Early Bird Heuristic (for Predicting Project Quality)](https://mlsanity.com/paper/2105.11082/) (2021)
- [SINGA-Easy: An Easy-to-Use Framework for MultiModal Analysis](https://mlsanity.com/paper/2108.02572/) (2021)
- [VisualTextRank: Unsupervised Graph-based Content Extraction for Automating Ad Text to Image Search](https://mlsanity.com/paper/2108.02725/) (2021)
- [Text is NOT Enough: Integrating Visual Impressions into Open-domain Dialogue Generation](https://mlsanity.com/paper/2109.05778/) (2021)
- [Discovering Personalized Semantics for Soft Attributes in Recommender Systems using Concept Activation Vectors](https://mlsanity.com/paper/2202.02830/) (2022)
- [On The Effectiveness of One-Class Support Vector Machine in Different Defect Prediction Scenarios](https://mlsanity.com/paper/2202.12074/) (2022)
- [Reinforced MOOCs Concept Recommendation in Heterogeneous Information Networks](https://mlsanity.com/paper/2203.11011/) (2022)
- [Explanation-Guided Fairness Testing through Genetic Algorithm](https://mlsanity.com/paper/2205.08335/) (2022)
- [Using Graph Neural Networks for Program Termination](https://mlsanity.com/paper/2207.14648/) (2022)
- [Less, but Stronger: On the Value of Strong Heuristics in Semi-supervised Learning for Software Analytics](https://mlsanity.com/paper/2302.01997/) (2023)
- [Tribe or Not? Critical Inspection of Group Differences Using TribalGram](https://mlsanity.com/paper/2303.09664/) (2023)

## Abstract (arXiv)

We report and fix an important systematic error in prior studies that ranked classifiers for software analytics. Those studies did not (a) assess classifiers on multiple criteria and they did not (b) study how variations in the data affect the results. Hence, this paper applies (a) multi-criteria tests while (b) fixing the weaker regions of the training data (using SMOTUNED, which is a self-tuning version of SMOTE). This approach leads to dramatically large increases in software defect predictions. When applied in a 5*5 cross-validation study for 3,681 JAVA classes (containing over a million lines of code) from open source systems, SMOTUNED increased AUC and recall by 60% and 20% respectively. These improvements are independent of the classifier used to predict for quality. Same kind of pattern (improvement) was observed when a comparative analysis of SMOTE and SMOTUNED was done against the most recent class imbalance technique. In conclusion, for software analytic tasks like defect prediction, (1) data pre-processing can be more important than classifier choice, (2) ranking studies are incomplete without such pre-processing, and (3) SMOTUNED is a promising candidate for pre-processing.