# Training Gradient Boosted Decision Trees on Tabular Data Containing Label Noise for Classification Tasks

arXiv: https://arxiv.org/abs/2409.08647 · HTML: https://mlsanity.com/paper/2409.08647/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 17 of 17)

- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)
- [Identifying Mislabeled Data using the Area Under the Margin Ranking](https://mlsanity.com/paper/2001.10528/) (2020)
- [Decoupling "when to update" from "how to update"](https://mlsanity.com/paper/1706.02613/) (2017)
- [Learning to Learn from Noisy Labeled Data](https://mlsanity.com/paper/1812.05214/) (2018)
- [Learning From Noisy Large-Scale Datasets With Minimal Supervision](https://mlsanity.com/paper/1701.01619/) (2017)
- [Learning with Feature-Dependent Label Noise: A Progressive Approach](https://mlsanity.com/paper/2103.07756/) (2021)
- [Error-Bounded Correction of Noisy Labels](https://mlsanity.com/paper/2011.10077/) (2020)
- [Learning Deep Networks from Noisy Labels with Dropout Regularization](https://mlsanity.com/paper/1705.03419/) (2017)
- [Two Wrongs Don't Make a Right: Combating Confirmation Bias in Learning with Label Noise](https://mlsanity.com/paper/2112.02960/) (2021)
- [Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees](https://mlsanity.com/paper/2205.00359/) (2022)
- [Improving Data Quality with Training Dynamics of Gradient Boosting Decision Trees](https://mlsanity.com/paper/2210.11327/) (2022)

## Built on by (most cited first, 1 of 1)

- [Mislabeled examples detection viewed as probing machine learning models: concepts, survey and extensive benchmark](https://mlsanity.com/paper/2410.15772/) (2024)

## Abstract (arXiv)

Label noise, which refers to the mislabeling of instances in a dataset, can significantly impair classifier performance, increase model complexity, and affect feature selection. While most research has concentrated on deep neural networks for image and text data, this study explores the impact of label noise on gradient-boosted decision trees (GBDTs), the leading algorithm for tabular data. This research fills a gap by examining the robustness of GBDTs to label noise, focusing on adapting two noise detection methods from deep learning for use with GBDTs and introducing a new detection method called Gradients. Additionally, we extend a method initially designed for GBDTs to incorporate relabeling. By using diverse datasets such as Covertype and Breast Cancer, we systematically introduce varying levels of label noise and evaluate the effectiveness of early stopping and noise detection methods in maintaining model performance. Our noise detection methods achieve state-of-the-art results, with a noise detection accuracy above 99% on the Adult dataset across all noise levels. This work enhances the understanding of label noise in GBDTs and provides a foundation for future research in noise detection and correction methods.