mlsanityData to 14 Oct 2026

Graph neural network distillation

98 papers0 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: graph, gnns, knowledge, gnn, distillation

Key facts. Graph neural network distillation: 98 papers on arXiv since 2021 (data through 14 Oct 2026). 31 of its 98 mapped papers (32%) cite “Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework” (2021, arXiv 2103.02885). 1 new paper 25 Sep–14 Oct.

What’s new

25 Sep – 14 Oct 2026

1 new since 25 Sep

  1. Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs

Foundations

Most cited by papers here

  1. Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework2021cited by 31 here
  2. On Representation Knowledge Distillation for Graph Neural Networks2021cited by 17 here
  3. Graph-Free Knowledge Distillation for Graph Neural Networks2021cited by 16 here
  4. Compressing Deep Graph Neural Networks via Adversarial Knowledge Distillation2022cited by 11 here
  5. SA-MLP: Distilling Graph Knowledge from GNNs into Structure-Aware MLP2022cited by 9 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2021Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Frameworkcited by 31 here
  2. 2022FreeKD: Free-direction Knowledge Distillation for Graph Neural Networkscited by 8 here
  3. 2022Boosting Graph Neural Networks via Adaptive Knowledge Distillationcited by 5 here
  4. 2023Knowledge Distillation on Graphs: A Surveycited by 8 here
  5. 2023VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPscited by 3 here
19 more it builds on, not in this chain
  1. 2021Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspectivecited by 3 here
  2. 2021Distilling Knowledge via Knowledge Reviewcited by 3 here
  3. 2021Graph-Free Knowledge Distillation for Graph Neural Networkscited by 16 here
  4. 2021Does Knowledge Distillation Really Work?cited by 3 here
  5. 2021Distilling Holistic Knowledge with Graph Neural Networkscited by 4 here
  6. 2021On Representation Knowledge Distillation for Graph Neural Networkscited by 17 here
  7. 2021Multi-task Self-distillation for Graph-based Semi-Supervised Learningcited by 6 here
  8. 2022Decoupled Knowledge Distillationcited by 3 here
  9. 2022Knowledge Distillation with the Reused Teacher Classifiercited by 4 here
  10. 2022Data-Free Adversarial Knowledge Distillation for Graph Neural Networkscited by 3 here
  11. 2022Compressing Deep Graph Neural Networks via Adversarial Knowledge Distillationcited by 11 here
  12. 2022Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillationcited by 4 here
  13. 2022Teaching Yourself: Graph Self-Distillation on Neighborhood for Node Classificationcited by 4 here
  14. 2022SA-MLP: Distilling Graph Knowledge from GNNs into Structure-Aware MLPcited by 9 here
  15. 2022T2-GNN: Graph Neural Networks for Graphs with Incomplete Features and Structure via Teacher-Student Distillationcited by 3 here
  16. 2023Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPscited by 4 here
  17. 2023Quantifying the Knowledge in GNNs for Reliable Distillation into MLPscited by 8 here
  18. 2024AdaGMLP: AdaBoosting GNN-to-MLP Knowledge Distillationcited by 3 here
  19. 2024Teach Harder, Learn Poorer: Rethinking Hard Sample Distillation for GNN-to-MLP Knowledge Distillationcited by 3 here

The 24 most cited of 99.

Browse

30 of 98, ranked 6–35

  1. 6Knowledge Distillation on Graphs: A Survey20238
  2. 7Quantifying the Knowledge in GNNs for Reliable Distillation into MLPs20238
  3. 8FreeKD: Free-direction Knowledge Distillation for Graph Neural Networks20228
  4. 9Multi-task Self-distillation for Graph-based Semi-Supervised Learning20216
  5. 10Boosting Graph Neural Networks via Adaptive Knowledge Distillation20225
  6. 11Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs20234
  7. 12Teaching Yourself: Graph Self-Distillation on Neighborhood for Node Classification20224
  8. 13Distilling Holistic Knowledge with Graph Neural Networks20214
  9. 14AdaGMLP: AdaBoosting GNN-to-MLP Knowledge Distillation20243
  10. 15Teach Harder, Learn Poorer: Rethinking Hard Sample Distillation for GNN-to-MLP Knowledge Distillation20243
All 30
  1. 16VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs20233
  2. 17Data-Free Adversarial Knowledge Distillation for Graph Neural Networks20223
  3. 18T2-GNN: Graph Neural Networks for Graphs with Incomplete Features and Structure via Teacher-Student Distillation20223
  4. 19Distilling Large Language Models for Text-Attributed Graph Learning20242
  5. 20Accelerating Molecular Graph Neural Networks via Knowledge Distillation20232
  6. 21The Devil is in the Data: Learning Fair Graph Neural Networks via Partial Knowledge Distillation20232
  7. 22Alignahead: Online Cross-Layer Knowledge Extraction on Graph Neural Networks20222
  8. 23Geometric Knowledge Distillation: Topology Compression for Graph Neural Networks20222
  9. 24Large Language Model Meets Graph Neural Network in Knowledge Distillation20241
  10. 25A Teacher-Free Graph Knowledge Distillation Framework with Dual Self-Distillation20241
  11. 26FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning20241
  12. 27Graph Knowledge Distillation to Mixture of Experts20241
  13. 28RELIANT: Fair Knowledge Distillation for Graph Neural Networks20231
  14. 29Graph-based Knowledge Distillation: A survey and experimental evaluation20231
  15. 30STRIDE: Structure and Embedding Distillation with Attention for Graph Neural Networks20231
  16. 31Bridging Classical and Quantum Machine Learning: Knowledge Transfer From Classical to Quantum Neural Networks Using Knowledge Distillation20231
  17. 32Geometer: Graph Few-Shot Class-Incremental Learning via Prototype Representation20221
  18. 33HIRE: Distilling High-order Relational Knowledge From Heterogeneous Graph Neural Networks20221
  19. 34GKD: Semi-supervised Graph Knowledge Distillation for Graph-Independent Inference20211
  20. 35Investigating Knowledge Distillation Through Neural Networks for Protein Binding Affinity Prediction20260