# ITBoost: Information-Theoretic Trust for Robust Boosting

arXiv: https://arxiv.org/abs/2605.04671 · HTML: https://mlsanity.com/paper/2605.04671/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 43 references that are in the map.
- First posted on arXiv 6 May 2026.

## Builds on (most cited first, 7 of 7)

- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [NGBoost: Natural Gradient Boosting for Probabilistic Prediction](https://mlsanity.com/paper/1910.03225/) (2019)
- [A Comparative Analysis of XGBoost](https://mlsanity.com/paper/1911.01914/) (2019)
- [Verifying Robustness of Gradient Boosted Models](https://mlsanity.com/paper/1906.10991/) (2019)
- [SPLBoost: An Improved Robust Boosting Algorithm Based on Self-paced Learning](https://mlsanity.com/paper/1706.06341/) (2017)

## Built on by (most cited first, 2 of 2)

- [When Does ℓ2-Boosting Overfit Benignly? High-Dimensional Risk Asymptotics and the ℓ1 Implicit Bias](https://mlsanity.com/paper/2605.06314/) (2026)
- [Reducing Learner Redundancy in Boosting via Residual Orthogonalization](https://mlsanity.com/paper/2606.17567/) (2026)

## Abstract (arXiv)

Gradient boosting remains a strong and widely used method for tabular data learning, but its performance often degrades when training labels are noisy. This behavior is largely related to the way boosting algorithms emphasize samples with large gradients, without explicitly accounting for whether such errors originate from informative hard cases or from unreliable labels. We address this issue by reconsidering how sample reliability is evaluated during boosting. Instead of relying on instantaneous error, we examine the evolution of each sample's residuals across iterations. Based on this insight, we propose Information-Theoretic Trust Boosting (ITBoost), which uses the Minimum Description Length principle to measure the complexity of residual trajectories. Samples whose residual patterns fluctuate in an irregular manner are treated as less trustworthy and are down-weighted during learning. Theoretically, we derive a tighter generalization bound for ITBoost under label noise. Empirical results on various tabular benchmarks indicate that ITBoost provides improved robustness in noisy environments over leading boosting and deep tabular models, while retaining best average performance on clean data.