mlsanityData to 7 Oct 2026

Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs

arXiv 2303.13763 ↗First posted 24 Mar 2023cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Distilling high-accuracy Graph Neural Networks (GNNs) to low-latency multilayer perceptions (MLPs) on graph tasks has become a hot research topic. However, conventional MLP learning relies almost exclusively on graph nodes and fails to effectively capture the graph structural information. Previous methods address this issue by processing graph edges into extra inputs for MLPs, but such graph structures may be unavailable for various scenarios. To this end, we propose Prototype-Guided Knowledge Distillation (PGKD), which does not require graph edges (edge-free setting) yet learns structure-aware MLPs. Our insight is to distill graph structural information from GNNs. Specifically, we first employ the class prototypes to analyze the impact of graph structures on GNN teachers, and then design two losses to distill such information from GNNs to MLPs. Experimental results on popular graph benchmarks demonstrate the effectiveness and robustness of the proposed PGKD.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 25 of its 42 references that are in the map. First posted on arXiv 24 Mar 2023.

Builds on

25 of 42 references.

References · 25

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2017Graph Attention Networkscited by 8,854
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  7. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  8. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  9. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
  10. 2019GNNExplainer: Generating Explanations for Graph Neural Networkscited by 735
  11. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
  12. 2017Few-Shot Learning with Graph Neural Networkscited by 324
13 more
  1. 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 309
  2. 2021Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillationcited by 151
  3. 2021Interpreting and Unifying Graph Neural Networks with An Optimization Frameworkcited by 131
  4. 2020A Unified View on Graph Neural Networks as Graph Signal Denoisingcited by 110
  5. 2020Graph Meta Learning via Local Subgraphscited by 89
  6. 2021Graph-MLP: Node Classification without Message Passing in Graphcited by 73
  7. 2019Graph Few-shot Learning via Knowledge Transfercited by 63
  8. 2022Federated Graph Neural Networks: Overview, Techniques and Challengescited by 52
  9. 2021Zero-shot Node Classification with Decomposed Graph Prototype Networkcited by 15
  10. 2022NOSMOG: Learning Noise-robust and Structure-aware MLPs on Graphscited by 14
  11. 2022SA-MLP: Distilling Graph Knowledge from GNNs into Structure-Aware MLPcited by 12
  12. 2022Teaching Yourself: Graph Self-Distillation on Neighborhood for Node Classificationcited by 6
  13. 2022PAGE: Prototype-Based Model-Level Explanations for Graph Neural Networkscited by 5

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 3.

7 papers.

  1. 2023Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting it into MLPs: An Effective GNN-to-MLP Distillation Frameworkcited by 24
  2. 2024Large Language Model Meets Graph Neural Network in Knowledge Distillationcited by 13
  3. 2023Weight-Inherited Distillation for Task-Agnostic BERT Compressioncited by 8
  4. 2024AdaGMLP: AdaBoosting GNN-to-MLP Knowledge Distillationcited by 6
  5. 2024LoCa: Logit Calibration for Knowledge Distillationcited by 2
  6. 2023Propagate & Distill: Towards Effective Graph Learners Using Propagation-Embracing MLPscited by 1
  7. 2023Unveiling the Unseen Potential of Graph Learning through MLPs: Effective Graph Learners Using Propagation-Embracing MLPscited by 0