# A Modified Bayesian Optimization based Hyper-Parameter Tuning Approach for Extreme Gradient Boosting

arXiv: https://arxiv.org/abs/2004.05041 · HTML: https://mlsanity.com/paper/2004.05041/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2020.

## Built on by (most cited first, 5 of 5)

- [Sequential Large Language Model-Based Hyper-parameter Optimization](https://mlsanity.com/paper/2410.20302/) (2024)
- [An Integrated Optimization and Machine Learning Models to Predict the Admission Status of Emergency Patients](https://mlsanity.com/paper/2202.09196/) (2022)
- [An Adaptive Simulated Annealing-Based Machine Learning Approach for Developing an E-Triage Tool for Hospital Emergency Operations](https://mlsanity.com/paper/2212.11892/) (2022)
- [Predicting Mutual Funds' Performance using Deep Learning and Ensemble Techniques](https://mlsanity.com/paper/2209.09649/) (2022)
- [Ideological Stance Detection in a Low-Resource Language: Polarization in Bangladeshi Public vs Private University Discourse on Social Media](https://mlsanity.com/paper/2610.04401/) (2026)

## Abstract (arXiv)

It is already reported in the literature that the performance of a machine learning algorithm is greatly impacted by performing proper Hyper-Parameter optimization. One of the ways to perform Hyper-Parameter optimization is by manual search but that is time consuming. Some of the common approaches for performing Hyper-Parameter optimization are Grid search Random search and Bayesian optimization using Hyperopt. In this paper, we propose a brand new approach for hyperparameter improvement i.e. Randomized-Hyperopt and then tune the hyperparameters of the XGBoost i.e. the Extreme Gradient Boosting algorithm on ten datasets by applying Random search, Randomized-Hyperopt, Hyperopt and Grid Search. The performances of each of these four techniques were compared by taking both the prediction accuracy and the execution time into consideration. We find that the Randomized-Hyperopt performs better than the other three conventional methods for hyper-paramter optimization of XGBoost.