# Boost then Convolve: Gradient Boosting Meets Graph Neural Networks

arXiv: https://arxiv.org/abs/2101.08543 · HTML: https://mlsanity.com/paper/2101.08543/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 21 Jan 2021.

## Builds on (most cited first, 30 of 32)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](https://mlsanity.com/paper/1810.05997/) (2018)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Multi-scale Attributed Node Embedding](https://mlsanity.com/paper/1909.13021/) (2019)
- [Few-Shot Learning with Graph Neural Networks](https://mlsanity.com/paper/1711.04043/) (2017)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [Reinforcement Learning for Combinatorial Optimization: A Survey](https://mlsanity.com/paper/2003.03600/) (2020)
- [Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data](https://mlsanity.com/paper/1909.06312/) (2019)
- [What graph neural networks cannot learn: depth vs width](https://mlsanity.com/paper/1907.03199/) (2019)
- [Universal Invariant and Equivariant Graph Neural Networks](https://mlsanity.com/paper/1905.04943/) (2019)
- [Sparse Sequence-to-Sequence Models](https://mlsanity.com/paper/1905.05702/) (2019)
- [On the Universality of Invariant Networks](https://mlsanity.com/paper/1901.09342/) (2019)
- [Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data](https://mlsanity.com/paper/1910.08233/) (2019)
- [Attention-based Graph Neural Network for Semi-supervised Learning](https://mlsanity.com/paper/1803.03735/) (2018)
- [Deep Graph Matching Consensus](https://mlsanity.com/paper/2001.09621/) (2020)
- [A Comparative Analysis of XGBoost](https://mlsanity.com/paper/1911.01914/) (2019)
- [Deep Neural Decision Trees](https://mlsanity.com/paper/1806.06988/) (2018)
- [The Tree Ensemble Layer: Differentiability meets Conditional Computation](https://mlsanity.com/paper/2002.07772/) (2020)
- [VERSE: Versatile Graph Embeddings from Similarity Measures](https://mlsanity.com/paper/1803.04742/) (2018)
- [Deep Forest](https://mlsanity.com/paper/1702.08835/) (2017)
- [Gradient Boosting Neural Networks: GrowNet](https://mlsanity.com/paper/2002.07971/) (2020)
- [Heterogeneous Deep Graph Infomax](https://mlsanity.com/paper/1911.08538/) (2019)
- [AdaGCN: Adaboosting Graph Convolutional Networks into Deep Models](https://mlsanity.com/paper/1908.05081/) (2019)
- [Abstract Diagrammatic Reasoning with Multiplex Graph Networks](https://mlsanity.com/paper/2006.11197/) (2020)

## Built on by (most cited first, 30 of 44)

- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024)
- [Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation](https://mlsanity.com/paper/2110.08727/) (2021)
- [ReconBoost: Boosting Can Achieve Modality Reconcilement](https://mlsanity.com/paper/2405.09321/) (2024)
- [GADBench: Revisiting and Benchmarking Supervised Graph Anomaly Detection](https://mlsanity.com/paper/2306.12251/) (2023)
- [ARC: A Generalist Graph Anomaly Detector with In-Context Learning](https://mlsanity.com/paper/2405.16771/) (2024)
- [Turning Tabular Foundation Models into Graph Foundation Models](https://mlsanity.com/paper/2508.20906/) (2025)
- [PyTorch Frame: A Modular Framework for Multi-Modal Tabular Learning](https://mlsanity.com/paper/2404.00776/) (2024)
- [GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Data](https://mlsanity.com/paper/2409.14500/) (2024)
- [A Survey on Deep Tabular Learning](https://mlsanity.com/paper/2410.12034/) (2024)
- [Motif Graph Neural Network](https://mlsanity.com/paper/2112.14900/) (2021)
- [Why Propagate Alone? Parallel Use of Labels and Features on Graphs](https://mlsanity.com/paper/2110.07190/) (2021)
- [Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features](https://mlsanity.com/paper/2110.13413/) (2021)
- [From Few-Shot to Zero-Shot: Towards Generalist Graph Anomaly Detection](https://mlsanity.com/paper/2602.18793/) (2026)
- [Simplifying Node Classification on Heterophilous Graphs with Compatible Label Propagation](https://mlsanity.com/paper/2205.09389/) (2022)
- [Decouple Graph Neural Networks: Train Multiple Simple GNNs Simultaneously Instead of One](https://mlsanity.com/paper/2304.10126/) (2023)
- [Interpretable Graph Neural Networks for Tabular Data](https://mlsanity.com/paper/2308.08945/) (2023)
- [Gradient Boosting Trees and Large Language Models for Tabular Data Few-Shot Learning](https://mlsanity.com/paper/2411.04324/) (2024)
- [Graph Learning Indexer: A Contributor-Friendly and Metadata-Rich Platform for Graph Learning Benchmarks](https://mlsanity.com/paper/2212.04537/) (2022)
- [Rethinking Feature Alignment in Generalist Graph Anomaly Detection: A Relational Fingerprint-based Approach](https://mlsanity.com/paper/2605.25429/) (2026)
- [TREE-G: Decision Trees Contesting Graph Neural Networks](https://mlsanity.com/paper/2207.02760/) (2022)
- [ColdGuess: A General and Effective Relational Graph Convolutional Network to Tackle Cold Start Cases](https://mlsanity.com/paper/2205.12318/) (2022)
- [Propagation on Multi-relational Graphs for Node Regression](https://mlsanity.com/paper/2110.08185/) (2021)
- [Dimensionality Reduction Meets Message Passing for Graph Node Embeddings](https://mlsanity.com/paper/2202.00408/) (2022)
- [High Performance of Gradient Boosting in Binding Affinity Prediction](https://mlsanity.com/paper/2205.07023/) (2022)
- [A Robust Stacking Framework for Training Deep Graph Models with Multifaceted Node Features](https://mlsanity.com/paper/2206.08473/) (2022)
- [New drugs and stock market: how to predict pharma market reaction to clinical trial announcements](https://mlsanity.com/paper/2208.07248/) (2022)
- [Pretrained deep models outperform GBDTs in Learning-To-Rank under label scarcity](https://mlsanity.com/paper/2308.00177/) (2023)
- [A Model-Agnostic Graph Neural Network for Integrating Local and Global Information](https://mlsanity.com/paper/2309.13459/) (2023)

## Abstract (arXiv)

Graph neural networks (GNNs) are powerful models that have been successful in various graph representation learning tasks. Whereas gradient boosted decision trees (GBDT) often outperform other machine learning methods when faced with heterogeneous tabular data. But what approach should be used for graphs with tabular node features? Previous GNN models have mostly focused on networks with homogeneous sparse features and, as we show, are suboptimal in the heterogeneous setting. In this work, we propose a novel architecture that trains GBDT and GNN jointly to get the best of both worlds: the GBDT model deals with heterogeneous features, while GNN accounts for the graph structure. Our model benefits from end-to-end optimization by allowing new trees to fit the gradient updates of GNN. With an extensive experimental comparison to the leading GBDT and GNN models, we demonstrate a significant increase in performance on a variety of graphs with tabular features. The code is available: https://github.com/nd7141/bgnn.