# Benchmarking state-of-the-art gradient boosting algorithms for classification

arXiv: https://arxiv.org/abs/2305.17094 · HTML: https://mlsanity.com/paper/2305.17094/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 23 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 3 of 3)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [A Comparative Analysis of XGBoost](https://mlsanity.com/paper/1911.01914/) (2019)
- [Hyperboost: Hyperparameter Optimization by Gradient Boosting surrogate models](https://mlsanity.com/paper/2101.02289/) (2021)

## Built on by (most cited first, 4 of 4)

- [Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Trees](https://mlsanity.com/paper/2309.09968/) (2023)
- [Wasserstein Gradient Boosting: A Framework for Distribution-Valued Supervised Learning](https://mlsanity.com/paper/2405.09536/) (2024)
- [Causal multi-modal AI for personalized chemosensitivity prediction](https://mlsanity.com/paper/2609.13567/) (2026)
- [Boosting Predictive Performance on Tabular Data through Data Augmentation with Latent-Space Flow-Based Diffusion](https://mlsanity.com/paper/2511.16571/) (2025)

## Abstract (arXiv)

This work explores the use of gradient boosting in the context of classification. Four popular implementations, including original GBM algorithm and selected state-of-the-art gradient boosting frameworks (i.e. XGBoost, LightGBM and CatBoost), have been thoroughly compared on several publicly available real-world datasets of sufficient diversity. In the study, special emphasis was placed on hyperparameter optimization, specifically comparing two tuning strategies, i.e. randomized search and Bayesian optimization using the Tree-stuctured Parzen Estimator. The performance of considered methods was investigated in terms of common classification accuracy metrics as well as runtime and tuning time. Additionally, obtained results have been validated using appropriate statistical testing. An attempt was made to indicate a gradient boosting variant showing the right balance between effectiveness, reliability and ease of use.