# AdaGMLP: AdaBoosting GNN-to-MLP Knowledge Distillation

arXiv: https://arxiv.org/abs/2405.14307 · HTML: https://mlsanity.com/paper/2405.14307/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Graph neural network distillation, one of 12 directions in Knowledge distillation.
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 23 May 2024.

## Placed in

- Knowledge distillation / [Graph neural network distillation](https://mlsanity.com/distill/graph-neural-network-distillation/)

## Builds on (most cited first, 21 of 21)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](https://mlsanity.com/paper/1810.05997/) (2018)
- [Graph Neural Networks for Social Recommendation](https://mlsanity.com/paper/1902.07243/) (2019)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation](https://mlsanity.com/paper/2110.08727/) (2021)
- [Distilling Knowledge from Graph Convolutional Networks](https://mlsanity.com/paper/2003.10477/) (2020)
- [Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework](https://mlsanity.com/paper/2103.02885/) (2021)
- [On Self-Distilling Graph Neural Network](https://mlsanity.com/paper/2011.02255/) (2020)
- [Quantifying the Knowledge in GNNs for Reliable Distillation into MLPs](https://mlsanity.com/paper/2306.05628/) (2023)
- [Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting it into MLPs: An Effective GNN-to-MLP Distillation Framework](https://mlsanity.com/paper/2305.10758/) (2023)
- [SA-MLP: Distilling Graph Knowledge from GNNs into Structure-Aware MLP](https://mlsanity.com/paper/2210.09609/) (2022)
- [NodeMixup: Tackling Under-Reaching for Graph Neural Networks](https://mlsanity.com/paper/2312.13032/) (2023)
- [Multi-task Self-distillation for Graph-based Semi-Supervised Learning](https://mlsanity.com/paper/2112.01174/) (2021)
- [Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs](https://mlsanity.com/paper/2303.13763/) (2023)
- [Teaching Yourself: Graph Self-Distillation on Neighborhood for Node Classification](https://mlsanity.com/paper/2210.02097/) (2022)

## Built on by (most cited first, 6 of 6)

- [GraphSOS: Graph Sampling and Order Selection to Help LLMs Understand Graphs Better](https://mlsanity.com/paper/2501.14427/) (2025)
- [DSBD: Dual-Aligned Structural Basis Distillation for Graph Domain Adaptation](https://mlsanity.com/paper/2604.03154/) (2026)
- [ProGMLP: A Progressive Framework for GNN-to-MLP Knowledge Distillation with Efficient Trade-offs](https://mlsanity.com/paper/2507.19031/) (2025)
- [InfGraND: An Influence-Guided GNN-to-MLP Knowledge Distillation](https://mlsanity.com/paper/2601.08033/) (2026)
- [Heuristic Methods are Good Teachers to Distill MLPs for Graph Link Prediction](https://mlsanity.com/paper/2504.06193/) (2025)
- [Towards Pre-trained Graph Condensation via Optimal Transport](https://mlsanity.com/paper/2509.14722/) (2025)

## Abstract (arXiv)

Graph Neural Networks (GNNs) have revolutionized graph-based machine learning, but their heavy computational demands pose challenges for latency-sensitive edge devices in practical industrial applications. In response, a new wave of methods, collectively known as GNN-to-MLP Knowledge Distillation, has emerged. They aim to transfer GNN-learned knowledge to a more efficient MLP student, which offers faster, resource-efficient inference while maintaining competitive performance compared to GNNs. However, these methods face significant challenges in situations with insufficient training data and incomplete test data, limiting their applicability in real-world applications. To address these challenges, we propose AdaGMLP, an AdaBoosting GNN-to-MLP Knowledge Distillation framework. It leverages an ensemble of diverse MLP students trained on different subsets of labeled nodes, addressing the issue of insufficient training data. Additionally, it incorporates a Node Alignment technique for robust predictions on test data with missing or incomplete features. Our experiments on seven benchmark datasets with different settings demonstrate that AdaGMLP outperforms existing G2M methods, making it suitable for a wide range of latency-sensitive real-world applications. We have submitted our code to the GitHub repository (https://github.com/WeigangLu/AdaGMLP-KDD24).