mlsanityData to 14 Oct 2026

Boost then Convolve: Gradient Boosting Meets Graph Neural Networks

arXiv 2101.08543 ↗First posted 21 Jan 2021cs.LG cs.AI cs.SIcited by 44

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) are powerful models that have been successful in various graph representation learning tasks. Whereas gradient boosted decision trees (GBDT) often outperform other machine learning methods when faced with heterogeneous tabular data. But what approach should be used for graphs with tabular node features? Previous GNN models have mostly focused on networks with homogeneous sparse features and, as we show, are suboptimal in the heterogeneous setting. In this work, we propose a novel architecture that trains GBDT and GNN jointly to get the best of both worlds: the GBDT model deals with heterogeneous features, while GNN accounts for the graph structure. Our model benefits from end-to-end optimization by allowing new trees to fit the gradient updates of GNN. With an extensive experimental comparison to the leading GBDT and GNN models, we demonstrate a significant increase in performance on a variety of graphs with tabular features. The code is available: https://github.com/nd7141/bgnn.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Builds on 32 of its 45 references that are in the map. First posted on arXiv 21 Jan 2021.

Builds on

32 of 45 references.

References · 32

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  4. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  5. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  6. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  7. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 993
  8. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  9. 2019TabNet: Attentive Interpretable Tabular Learningcited by 648
  10. 2019Multi-scale Attributed Node Embeddingcited by 548
  11. 2017Few-Shot Learning with Graph Neural Networkscited by 324
  12. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 320
20 more
  1. 2020Reinforcement Learning for Combinatorial Optimization: A Surveycited by 236
  2. 2019Neural Oblivious Decision Ensembles for Deep Learning on Tabular Datacited by 203
  3. 2019What graph neural networks cannot learn: depth vs widthcited by 201
  4. 2019Universal Invariant and Equivariant Graph Neural Networkscited by 183
  5. 2019Sparse Sequence-to-Sequence Modelscited by 170
  6. 2019On the Universality of Invariant Networkscited by 152
  7. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  8. 2018Attention-based Graph Neural Network for Semi-supervised Learningcited by 101
  9. 2020Deep Graph Matching Consensuscited by 85
  10. 2019A Comparative Analysis of XGBoostcited by 80
  11. 2018Deep Neural Decision Treescited by 72
  12. 2020The Tree Ensemble Layer: Differentiability meets Conditional Computationcited by 57
  13. 2018VERSE: Versatile Graph Embeddings from Similarity Measurescited by 54
  14. 2017Deep Forestcited by 46
  15. 2020Gradient Boosting Neural Networks: GrowNetcited by 46
  16. 2019Heterogeneous Deep Graph Infomaxcited by 42
  17. 2019AdaGCN: Adaboosting Graph Convolutional Networks into Deep Modelscited by 37
  18. 2020Abstract Diagrammatic Reasoning with Multiplex Graph Networkscited by 37
  19. 2018Multi-Layered Gradient Boosting Decision Treescited by 16
  20. 2020Residual Correlation in Graph Neural Network Regressioncited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 9; 2023: 5; 2024: 12; 2025: 2; 2026: 8.

44 papers.

Sort:

By area:

  1. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
  2. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 172
  3. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  4. 2021Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillationcited by 152
  5. 2024ReconBoost: Boosting Can Achieve Modality Reconcilementcited by 60
  6. 2023GADBench: Revisiting and Benchmarking Supervised Graph Anomaly Detectioncited by 58
  7. 2024ARC: A Generalist Graph Anomaly Detector with In-Context Learningcited by 48
  8. 2025Turning Tabular Foundation Models into Graph Foundation Modelscited by 35
  9. 2024PyTorch Frame: A Modular Framework for Multi-Modal Tabular Learningcited by 32
  10. 2024GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Datacited by 20
  11. 2024A Survey on Deep Tabular Learningcited by 18
  12. 2021Motif Graph Neural Networkcited by 10
32 more
  1. 2021Why Propagate Alone? Parallel Use of Labels and Features on Graphscited by 9
  2. 2021Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node featurescited by 9
  3. 2026From Few-Shot to Zero-Shot: Towards Generalist Graph Anomaly Detectioncited by 8
  4. 2022Simplifying Node Classification on Heterophilous Graphs with Compatible Label Propagationcited by 7
  5. 2023Decouple Graph Neural Networks: Train Multiple Simple GNNs Simultaneously Instead of Onecited by 7
  6. 2023Interpretable Graph Neural Networks for Tabular Datacited by 5
  7. 2024Gradient Boosting Trees and Large Language Models for Tabular Data Few-Shot Learningcited by 5
  8. 2022Graph Learning Indexer: A Contributor-Friendly and Metadata-Rich Platform for Graph Learning Benchmarkscited by 4
  9. 2026Rethinking Feature Alignment in Generalist Graph Anomaly Detection: A Relational Fingerprint-based Approachcited by 4
  10. 2022TREE-G: Decision Trees Contesting Graph Neural Networkscited by 3
  11. 2022ColdGuess: A General and Effective Relational Graph Convolutional Network to Tackle Cold Start Casescited by 2
  12. 2021Propagation on Multi-relational Graphs for Node Regressioncited by 1
  13. 2022Dimensionality Reduction Meets Message Passing for Graph Node Embeddingscited by 1
  14. 2022High Performance of Gradient Boosting in Binding Affinity Predictioncited by 1
  15. 2022A Robust Stacking Framework for Training Deep Graph Models with Multifaceted Node Featurescited by 1
  16. 2022New drugs and stock market: how to predict pharma market reaction to clinical trial announcementscited by 1
  17. 2023Pretrained deep models outperform GBDTs in Learning-To-Rank under label scarcitycited by 1
  18. 2023A Model-Agnostic Graph Neural Network for Integrating Local and Global Informationcited by 1
  19. 2024Gradient Boosting Reinforcement Learningcited by 1
  20. 2024TREB: a BERT attempt for imputing tabular data imputationcited by 1
  21. 2026Generalist Graph Anomaly Detection via Prototype-Based Distillationcited by 1
  22. 2021Learning Multi-Layered GBDT Via Back Propagationcited by 0
  23. 2021SStaGCN: Simplified stacking based graph convolutional networkscited by 0
  24. 2022High-Order Optimization of Gradient Boosted Decision Treescited by 0
  25. 2024Breast Cancer Classification Using Gradient Boosting Algorithms Focusing on Reducing the False Negative and SHAP for Explainabilitycited by 0
  26. 2024MBExplainer: Multilevel bandit-based explanations for downstream models with augmented graph embeddingscited by 0
  27. 2025Robust Anomaly Detection with Graph Neural Networks using Controllabilitycited by 0
  28. 2026Closing the gap on tabular data with Fourier and Implicit Categorical Featurescited by 0
  29. 2026A Zero-shot Generalized Graph Anomaly Detection Framework via Node Reconstructioncited by 0
  30. 2026ProTAGAD: A Foundation Model for TAG Anomaly Detection with Decoupled Topological and Textual Prototypescited by 0
  31. 2026Learning-Assisted Congestion-Aware Route Scheduling for Semiconductor Fab Material Control Systemscited by 0
  32. 2026Is Real-World Training Data Necessary for Generalist Graph Anomaly Detection?cited by 0