# Multi-Layered Gradient Boosting Decision Trees

arXiv: https://arxiv.org/abs/1806.00007 · HTML: https://mlsanity.com/paper/1806.00007/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2018.

## Built on by (most cited first, 16 of 16)

- [DANets: Deep Abstract Networks for Tabular Data Classification and Regression](https://mlsanity.com/paper/2112.02962/) (2021)
- [Boost then Convolve: Gradient Boosting Meets Graph Neural Networks](https://mlsanity.com/paper/2101.08543/) (2021)
- [T2G-Former: Organizing Tabular Features into Relation Graphs Promotes Heterogeneous Feature Interaction](https://mlsanity.com/paper/2211.16887/) (2022)
- [Representation Learning for Tabular Data: A Comprehensive Survey](https://mlsanity.com/paper/2504.16109/) (2025)
- [A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications](https://mlsanity.com/paper/2111.01968/) (2021)
- [Muddling Label Regularization: Deep Learning for Tabular Datasets](https://mlsanity.com/paper/2106.04462/) (2021)
- [AGBoost: Attention-based Modification of Gradient Boosting Machine](https://mlsanity.com/paper/2207.05724/) (2022)
- [Improve Deep Forest with Learnable Layerwise Augmentation Policy Schedule](https://mlsanity.com/paper/2309.09030/) (2023)
- [Efficient Click-Through Rate Prediction for Developing Countries via Tabular Learning](https://mlsanity.com/paper/2104.07553/) (2021)
- [Enhancing Transformers with Gradient Boosted Decision Trees for NLI Fine-Tuning](https://mlsanity.com/paper/2105.03791/) (2021)
- [A Framework for an Assessment of the Kernel-target Alignment in Tree Ensemble Kernel Learning](https://mlsanity.com/paper/2108.08752/) (2021)
- [Learning Multi-Layered GBDT Via Back Propagation](https://mlsanity.com/paper/2109.11863/) (2021)
- [A generalized decision tree ensemble based on the NeuralNetworks architecture: Distributed Gradient Boosting Forest (DGBF)](https://mlsanity.com/paper/2402.03386/) (2024)
- [A Methodology-Oriented Study of Catastrophic Forgetting in Incremental Deep Neural Networks](https://mlsanity.com/paper/2405.08015/) (2024)
- [An accuracy improving method for advertising click through rate prediction based on enhanced xDeepFM model](https://mlsanity.com/paper/2411.15223/) (2024)
- [Urban Boundaries, Social Barriers: A Benchmark and Vision-Centric Framework for Mapping Gated Communities and Equity Implications](https://mlsanity.com/paper/2609.03804/) (2026)

## Abstract (arXiv)

Multi-layered representation is believed to be the key ingredient of deep neural networks especially in cognitive tasks like computer vision. While non-differentiable models such as gradient boosting decision trees (GBDTs) are the dominant methods for modeling discrete or tabular data, they are hard to incorporate with such representation learning ability. In this work, we propose the multi-layered GBDT forest (mGBDTs), with an explicit emphasis on exploring the ability to learn hierarchical representations by stacking several layers of regression GBDTs as its building block. The model can be jointly trained by a variant of target propagation across layers, without the need to derive back-propagation nor differentiability. Experiments and visualizations confirmed the effectiveness of the model in terms of performance and representation learning ability.