# MT-GBM: A Multi-Task Gradient Boosting Machine with Shared Decision Trees

arXiv: https://arxiv.org/abs/2201.06239 · HTML: https://mlsanity.com/paper/2201.06239/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 29 references that are in the map.
- First posted on arXiv 17 Jan 2022.

## Builds on (most cited first, 6 of 6)

- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://mlsanity.com/paper/1901.11504/) (2019)
- [Identifying beneficial task relations for multi-task learning in deep neural networks](https://mlsanity.com/paper/1702.08303/) (2017)
- [Multi-Level Deep Cascade Trees for Conversion Rate Prediction in Recommendation System](https://mlsanity.com/paper/1805.09484/) (2018)
- [Interpretable Categorization of Heterogeneous Time Series Data](https://mlsanity.com/paper/1708.09121/) (2017)

## Built on by (most cited first, 4 of 4)

- [Scaling Up Diffusion and Flow-based XGBoost Models](https://mlsanity.com/paper/2408.16046/) (2024)
- [Multi-task CNN Behavioral Embedding Model For Transaction Fraud Detection](https://mlsanity.com/paper/2411.19457/) (2024)
- [Robust-Multi-Task Gradient Boosting](https://mlsanity.com/paper/2507.11411/) (2025)
- [Parallel gradient boosting for flexible estimation of conditional distributions](https://mlsanity.com/paper/2607.13550/) (2026)

## Abstract (arXiv)

Despite the success of deep learning in computer vision and natural language processing, Gradient Boosted Decision Tree (GBDT) is yet one of the most powerful tools for applications with tabular data such as e-commerce and FinTech. However, applying GBDT to multi-task learning is still a challenge. Unlike deep models that can jointly learn a shared latent representation across multiple tasks, GBDT can hardly learn a shared tree structure. In this paper, we propose Multi-task Gradient Boosting Machine (MT-GBM), a GBDT-based method for multi-task learning. The MT-GBM can find the shared tree structures and split branches according to multi-task losses. First, it assigns multiple outputs to each leaf node. Next, it computes the gradient corresponding to each output (task). Then, we also propose an algorithm to combine the gradients of all tasks and update the tree. Finally, we apply MT-GBM to LightGBM. Experiments show that our MT-GBM improves the performance of the main task significantly, which means the proposed MT-GBM is efficient and effective.