# Hyperparameter Optimization for Effort Estimation

arXiv: https://arxiv.org/abs/1805.00336 · HTML: https://mlsanity.com/paper/1805.00336/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2018.

## Built on by (most cited first, 13 of 13)

- [MMO: Meta Multi-Objectivization for Software Configuration Tuning](https://mlsanity.com/paper/2112.07303/) (2021)
- [FairMask: Better Fairness via Model-based Rebalancing of Protected Attributes](https://mlsanity.com/paper/2110.01109/) (2021)
- [The Weights can be Harmful: Pareto Search versus Weighted Search in Multi-Objective Search-Based Software Engineering](https://mlsanity.com/paper/2202.03728/) (2022)
- [Predicting Configuration Performance in Multiple Environments with Sequential Meta-learning](https://mlsanity.com/paper/2402.03183/) (2024)
- [Lifelong Dynamic Optimization for Self-Adaptive Systems: Fact or Fiction?](https://mlsanity.com/paper/2201.07096/) (2022)
- [Predicting Software Performance with Divide-and-Learn](https://mlsanity.com/paper/2306.06651/) (2023)
- [Does Configuration Encoding Matter in Learning Software Performance? An Empirical Study on Encoding Schemes](https://mlsanity.com/paper/2203.15988/) (2022)
- [Dividable Configuration Performance Learning](https://mlsanity.com/paper/2409.07629/) (2024)
- [Learning from Very Little Data: On the Value of Landscape Analysis for Predicting Software Project Health](https://mlsanity.com/paper/2301.06577/) (2023)
- [Software Engineering Principles for Fairer Systems: Experiments with GroupCART](https://mlsanity.com/paper/2504.12587/) (2025)
- [Evaluating Useful Surrogate Models for Configuration Tuning Beyond Accuracy: A Fitness Landscape Analysis Perspective](https://mlsanity.com/paper/2509.21945/) (2025)
- [Multifaceted Hierarchical Report Identification for Non-Functional Bugs in Deep Learning Frameworks](https://mlsanity.com/paper/2210.01855/) (2022)
- [Whence Is A Model Fair? Fixing Fairness Bugs via Propensity Score Matching](https://mlsanity.com/paper/2504.17066/) (2025)

## Abstract (arXiv)

Software analytics has been widely used in software engineering for many tasks such as generating effort estimates for software projects. One of the "black arts" of software analytics is tuning the parameters controlling a data mining algorithm. Such hyperparameter optimization has been widely studied in other software analytics domains (e.g. defect prediction and text mining) but, so far, has not been extensively explored for effort estimation. Accordingly, this paper seeks simple, automatic, effective and fast methods for finding good tunings for automatic software effort estimation. We introduce a hyperparameter optimization architecture called OIL (Optimized Inductive Learning). We test OIL on a wide range of hyperparameter optimizers using data from 945 software projects. After tuning, large improvements in effort estimation accuracy were observed (measured in terms of standardized accuracy). From those results, we recommend using regression trees (CART) tuned by different evolution combine with default analogy-based estimator. This particular combination of learner and optimizers often achieves in a few hours what other optimizers need days to weeks of CPU time to accomplish. An important part of this analysis is its reproducibility and refutability. All our scripts and data are on-line. It is hoped that this paper will prompt and enable much more research on better methods to tune software effort estimators.