# Tuning for Software Analytics: is it Really Necessary?

arXiv: https://arxiv.org/abs/1609.01759 · HTML: https://mlsanity.com/paper/1609.01759/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2016.

## Built on by (most cited first, 18 of 18)

- [The impact of feature importance methods on the interpretation of defect classifiers](https://mlsanity.com/paper/2202.02389/) (2022)
- [FRUGAL: Unlocking SSL for Software Analytics](https://mlsanity.com/paper/2108.09847/) (2021)
- [Fairness Concerns in App Reviews: A Study on AI-based Mobile Apps](https://mlsanity.com/paper/2401.08097/) (2024)
- [Learning from Very Little Data: On the Value of Landscape Analysis for Predicting Software Project Health](https://mlsanity.com/paper/2301.06577/) (2023)
- [Old but Gold: Reconsidering the value of feedforward learners for software analytics](https://mlsanity.com/paper/2101.06319/) (2021)
- [Is Hyper-Parameter Optimization Different for Software Analytics?](https://mlsanity.com/paper/2401.09622/) (2024)
- [Assessing the Early Bird Heuristic (for Predicting Project Quality)](https://mlsanity.com/paper/2105.11082/) (2021)
- [SINGA-Easy: An Easy-to-Use Framework for MultiModal Analysis](https://mlsanity.com/paper/2108.02572/) (2021)
- [VisualTextRank: Unsupervised Graph-based Content Extraction for Automating Ad Text to Image Search](https://mlsanity.com/paper/2108.02725/) (2021)
- [Text is NOT Enough: Integrating Visual Impressions into Open-domain Dialogue Generation](https://mlsanity.com/paper/2109.05778/) (2021)
- [Discovering Personalized Semantics for Soft Attributes in Recommender Systems using Concept Activation Vectors](https://mlsanity.com/paper/2202.02830/) (2022)
- [Reinforced MOOCs Concept Recommendation in Heterogeneous Information Networks](https://mlsanity.com/paper/2203.11011/) (2022)
- [Explanation-Guided Fairness Testing through Genetic Algorithm](https://mlsanity.com/paper/2205.08335/) (2022)
- [Using Graph Neural Networks for Program Termination](https://mlsanity.com/paper/2207.14648/) (2022)
- [Less, but Stronger: On the Value of Strong Heuristics in Semi-supervised Learning for Software Analytics](https://mlsanity.com/paper/2302.01997/) (2023)
- [Tribe or Not? Critical Inspection of Group Differences Using TribalGram](https://mlsanity.com/paper/2303.09664/) (2023)
- [A Feature-Driven Framework for Software Fault Prediction](https://mlsanity.com/paper/2605.17611/) (2026)
- [Is Model Instability just Noise to be Tolerated or a Property that can be Managed?](https://mlsanity.com/paper/2607.10420/) (2026)

## Abstract (arXiv)

Context: Data miners have been widely used in software engineering to, say, generate defect predictors from static code measures. Such static code defect predictors perform well compared to manual methods, and they are easy to use and useful to use. But one of the "black art" of data mining is setting the tunings that control the miner. Objective:We seek simple, automatic, and very effective method for finding those tunings. Method: For each experiment with different data sets (from open source JAVA systems), we ran differential evolution as anoptimizer to explore the tuning space (as a first step) then tested the tunings using hold-out data. Results: Contrary to our prior expectations, we found these tunings were remarkably simple: it only required tens, not thousands,of attempts to obtain very good results. For example, when learning software defect predictors, this method can quickly find tuningsthat alter detection precision from 0% to 60%. Conclusion: Since (1) the improvements are so large, and (2) the tuning is so simple, we need to change standard methods insoftware analytics. At least for defect prediction, it is no longer enough to just run a data miner and present the resultwithoutconducting a tuning optimization study. The implication for other kinds of analytics is now an open and pressing issue