# A Comparative study of Hyper-Parameter Optimization Tools

arXiv: https://arxiv.org/abs/2201.06433 · HTML: https://mlsanity.com/paper/2201.06433/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 19 references that are in the map.
- First posted on arXiv 17 Jan 2022.

## Builds on (most cited first, 4 of 4)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Easy Hyperparameter Search Using Optunity](https://mlsanity.com/paper/1412.1114/) (2014)
- [Bayesian Optimization for Machine Learning : A Practical Guidebook](https://mlsanity.com/paper/1612.04858/) (2016)

## Built on by (most cited first, 10 of 10)

- [RUMBoost: Gradient Boosted Random Utility Models](https://mlsanity.com/paper/2401.11954/) (2024)
- [Advancing Machine Learning in Industry 4.0: Benchmark Framework for Rare-event Prediction in Chemical Processes](https://mlsanity.com/paper/2409.00485/) (2024)
- [Experimental Evaluation of Machine Learning Models for Goal-oriented Customer Service Chatbot with Pipeline Architecture](https://mlsanity.com/paper/2409.18568/) (2024)
- [Ethical Fairness in Ubiquitous Health Sensing without Known Attributes](https://mlsanity.com/paper/2603.13373/) (2026)
- [Hyperparameter Optimization of Generative Adversarial Network Models for High-Energy Physics Simulations](https://mlsanity.com/paper/2208.07715/) (2022)
- [FORLORN: A Framework for Comparing Offline Methods and Reinforcement Learning for Optimization of RAN Parameters](https://mlsanity.com/paper/2209.13540/) (2022)
- [A PSO Based Method to Generate Actionable Counterfactuals for High Dimensional Data](https://mlsanity.com/paper/2311.12825/) (2023)
- [Testing the Efficacy of Hyperparameter Optimization Algorithms in Short-Term Load Forecasting](https://mlsanity.com/paper/2410.15047/) (2024)
- [AutoMR: A Universal Time Series Motion Recognition Pipeline](https://mlsanity.com/paper/2502.15228/) (2025)
- [Balancing Sparse RNNs with Hyperparameterization Benefiting Meta-Learning](https://mlsanity.com/paper/2509.15057/) (2025)

## Abstract (arXiv)

Most of the machine learning models have associated hyper-parameters along with their parameters. While the algorithm gives the solution for parameters, its utility for model performance is highly dependent on the choice of hyperparameters. For a robust performance of a model, it is necessary to find out the right hyper-parameter combination. Hyper-parameter optimization (HPO) is a systematic process that helps in finding the right values for them. The conventional methods for this purpose are grid search and random search and both methods create issues in industrial-scale applications. Hence a set of strategies have been recently proposed based on Bayesian optimization and evolutionary algorithm principles that help in runtime issues in a production environment and robust performance. In this paper, we compare the performance of four python libraries, namely Optuna, Hyper-opt, Optunity, and sequential model-based algorithm configuration (SMAC) that has been proposed for hyper-parameter optimization. The performance of these tools is tested using two benchmarks. The first one is to solve a combined algorithm selection and hyper-parameter optimization (CASH) problem The second one is the NeurIPS black-box optimization challenge in which a multilayer perception (MLP) architecture has to be chosen from a set of related architecture constraints and hyper-parameters. The benchmarking is done with six real-world datasets. From the experiments, we found that Optuna has better performance for CASH problem and HyperOpt for MLP problem.