# Robust-GBDT: GBDT with Nonconvex Loss for Tabular Classification in the Presence of Label Noise and Class Imbalance

arXiv: https://arxiv.org/abs/2310.05067 · HTML: https://mlsanity.com/paper/2310.05067/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 44 references that are in the map.
- First posted on arXiv 8 Oct 2023.

## Builds on (most cited first, 16 of 16)

- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates](https://mlsanity.com/paper/1910.03231/) (2019)
- [Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning](https://mlsanity.com/paper/2006.07805/) (2020)
- [Curriculum Loss: Robust Learning and Generalization against Label Corruption](https://mlsanity.com/paper/1905.10045/) (2019)
- [Robust Bi-Tempered Logistic Loss Based on Bregman Divergences](https://mlsanity.com/paper/1906.03361/) (2019)
- [GBDT-MO: Gradient Boosted Decision Trees for Multiple Outputs](https://mlsanity.com/paper/1909.04373/) (2019)
- [Boosting in the presence of outliers: adaptive classification with non-convex loss functions](https://mlsanity.com/paper/1510.01064/) (2015)
- [SPLBoost: An Improved Robust Boosting Algorithm Based on Self-paced Learning](https://mlsanity.com/paper/1706.06341/) (2017)
- [TRBoost: A Generic Gradient Boosting Machine based on Trust-region Method](https://mlsanity.com/paper/2209.13791/) (2022)

## Abstract (arXiv)

Dealing with label noise in tabular classification tasks poses a persistent challenge in machine learning. While robust boosting methods have shown promise in binary classification, their effectiveness in complex, multi-class scenarios is often limited. Additionally, issues like imbalanced datasets, missing values, and computational inefficiencies further complicate their practical utility. This study introduces Robust-GBDT, a groundbreaking approach that combines the power of Gradient Boosted Decision Trees (GBDT) with the resilience of nonconvex loss functions against label noise. By leveraging local convexity within specific regions, Robust-GBDT demonstrates unprecedented robustness, challenging conventional wisdom. Through seamless integration of advanced GBDT with a novel Robust Focal Loss tailored for class imbalance, Robust-GBDT significantly enhances generalization capabilities, particularly in noisy and imbalanced datasets. Notably, its user-friendly design facilitates integration with existing open-source code, enhancing computational efficiency and scalability. Extensive experiments validate Robust-GBDT's superiority over other noise-robust methods, establishing a new standard for accurate classification amidst label noise. This research heralds a paradigm shift in machine learning, paving the way for a new era of robust and precise classification across diverse real-world applications.