# Robust-Multi-Task Gradient Boosting

arXiv: https://arxiv.org/abs/2507.11411 · HTML: https://mlsanity.com/paper/2507.11411/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 96 references that are in the map.
- First posted on arXiv 15 Jul 2025.

## Builds on (most cited first, 28 of 28)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [Efficiently Sampling Functions from Gaussian Process Posteriors](https://mlsanity.com/paper/2002.09309/) (2020)
- [When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications](https://mlsanity.com/paper/2310.18339/) (2023)
- [A Comparative Analysis of XGBoost](https://mlsanity.com/paper/1911.01914/) (2019)
- [Multitask Vision-Language Prompt Tuning](https://mlsanity.com/paper/2211.11720/) (2022)
- [MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learning](https://mlsanity.com/paper/2403.20320/) (2024)
- [Dynamic Task Weighting Methods for Multi-task Networks in Autonomous Driving Systems](https://mlsanity.com/paper/2001.02223/) (2020)
- [Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning](https://mlsanity.com/paper/2106.02705/) (2021)
- [A review of ensemble learning and data augmentation models for class imbalanced problems: combination, implementation and evaluation](https://mlsanity.com/paper/2304.02858/) (2023)
- [Taking Advantage of Multitask Learning for Fair Classification](https://mlsanity.com/paper/1810.08683/) (2018)
- [Heterogeneous Multi-task Learning with Expert Diversity](https://mlsanity.com/paper/2106.10595/) (2021)
- [GBDT-MO: Gradient Boosted Decision Trees for Multiple Outputs](https://mlsanity.com/paper/1909.04373/) (2019)
- [Window-Based Early-Exit Cascades for Uncertainty Estimation: When Deep Ensembles are More Efficient than Single Models](https://mlsanity.com/paper/2303.08010/) (2023)
- [Missing Value Estimation using Clustering and Deep Learning within Multiple Imputation Framework](https://mlsanity.com/paper/2202.13734/) (2022)
- [Heterogeneous Multi-task Learning for Human Pose Estimation with Deep Convolutional Neural Network](https://mlsanity.com/paper/1406.3474/) (2014)
- [Efficient Output Kernel Learning for Multiple Tasks](https://mlsanity.com/paper/1511.05706/) (2015)
- [Consistent Multitask Learning with Nonlinear Output Relations](https://mlsanity.com/paper/1705.08118/) (2017)
- [When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review](https://mlsanity.com/paper/2307.14382/) (2023)
- [MT-GBM: A Multi-Task Gradient Boosting Machine with Shared Decision Trees](https://mlsanity.com/paper/2201.06239/) (2022)
- [Task-wise Split Gradient Boosting Trees for Multi-center Diabetes Prediction](https://mlsanity.com/paper/2108.07107/) (2021)
- [Condensed Gradient Boosting](https://mlsanity.com/paper/2211.14599/) (2022)
- [Interpretable MTL from Heterogeneous Domains using Boosted Tree](https://mlsanity.com/paper/2003.07077/) (2020)

## Built on by (most cited first, 1 of 1)

- [Robust multi-task boosting using clustering and local ensembling](https://mlsanity.com/paper/2602.14231/) (2026)

## Abstract (arXiv)

Multi-task learning (MTL) has shown effectiveness in exploiting shared information across tasks to improve generalization. MTL assumes tasks share similarities that can improve performance. In addition, boosting algorithms have demonstrated exceptional performance across diverse learning problems, primarily due to their ability to focus on hard-to-learn instances and iteratively reduce residual errors. This makes them a promising approach for learning multi-task problems. However, real-world MTL scenarios often involve tasks that are not well-aligned (known as outlier or adversarial tasks), which do not share beneficial similarities with others and can, in fact, deteriorate the performance of the overall model. To overcome this challenge, we propose Robust-Multi-Task Gradient Boosting (R-MTGB), a novel boosting framework that explicitly models and adapts to task heterogeneity during training. R-MTGB structures the learning process into three sequential blocks: (1) learning shared patterns, (2) partitioning tasks into outliers and non-outliers with regularized parameters, and (3) fine-tuning task-specific predictors. This architecture enables R-MTGB to automatically detect and penalize outlier tasks while promoting effective knowledge transfer among related tasks. Our method integrates these mechanisms seamlessly within gradient boosting, allowing robust handling of noisy or adversarial tasks without sacrificing accuracy. Extensive experiments on both synthetic benchmarks and real-world datasets demonstrate that our approach successfully isolates outliers, transfers knowledge, and consistently reduces prediction errors for each task individually, and achieves overall performance gains across all tasks. These results highlight robustness, adaptability, and reliable convergence of R-MTGB in challenging MTL environments.