# Improving GBDT Performance on Imbalanced Datasets: An Empirical Study of Class-Balanced Loss Functions

arXiv: https://arxiv.org/abs/2407.14381 · HTML: https://mlsanity.com/paper/2407.14381/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 19 Jul 2024.

## Builds on (most cited first, 17 of 17)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification](https://mlsanity.com/paper/1602.02068/) (2016)
- [Equalization Loss for Long-Tailed Object Recognition](https://mlsanity.com/paper/2003.05176/) (2020)
- [Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets](https://mlsanity.com/paper/2007.09654/) (2020)
- [Influence-Balanced Loss for Imbalanced Visual Classification](https://mlsanity.com/paper/2110.02444/) (2021)
- [Equalized Focal Loss for Dense Long-Tailed Object Detection](https://mlsanity.com/paper/2201.02593/) (2022)
- [A survey on learning from imbalanced data streams: taxonomy, challenges, empirical study, and reproducible experimental framework](https://mlsanity.com/paper/2204.03719/) (2022)
- [A review of ensemble learning and data augmentation models for class imbalanced problems: combination, implementation and evaluation](https://mlsanity.com/paper/2304.02858/) (2023)
- [Imbalance-XGBoost: Leveraging Weighted and Focal Losses for Binary Label-Imbalanced Classification with XGBoost](https://mlsanity.com/paper/1908.01672/) (2019)
- [NCART: Neural Classification and Regression Tree for Tabular Data](https://mlsanity.com/paper/2307.12198/) (2023)
- [SketchBoost: Fast Gradient Boosted Decision Tree for Multioutput Problems](https://mlsanity.com/paper/2211.12858/) (2022)
- [TRBoost: A Generic Gradient Boosting Machine based on Trust-region Method](https://mlsanity.com/paper/2209.13791/) (2022)

## Abstract (arXiv)

Class imbalance remains a significant challenge in machine learning, particularly for tabular data classification tasks. While Gradient Boosting Decision Trees (GBDT) models have proven highly effective for such tasks, their performance can be compromised when dealing with imbalanced datasets. This paper presents the first comprehensive study on adapting class-balanced loss functions to three GBDT algorithms across various tabular classification tasks, including binary, multi-class, and multi-label classification. We conduct extensive experiments on multiple datasets to evaluate the impact of class-balanced losses on different GBDT models, establishing a valuable benchmark. Our results demonstrate the potential of class-balanced loss functions to enhance GBDT performance on imbalanced datasets, offering a robust approach for practitioners facing class imbalance challenges in real-world applications. Additionally, we introduce a Python package that facilitates the integration of class-balanced loss functions into GBDT workflows, making these advanced techniques accessible to a wider audience.