# GKD: Semi-supervised Graph Knowledge Distillation for Graph-Independent Inference

arXiv: https://arxiv.org/abs/2104.03597 · HTML: https://mlsanity.com/paper/2104.03597/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Graph neural network distillation, one of 12 directions in Knowledge distillation.
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Placed in

- Knowledge distillation / [Graph neural network distillation](https://mlsanity.com/distill/graph-neural-network-distillation/)

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Disease Prediction using Graph Convolutional Networks: Application to Autism Spectrum Disorder and Alzheimer's Disease](https://mlsanity.com/paper/1806.01738/) (2018)
- [Spectral Graph Convolutions for Population-based Disease Prediction](https://mlsanity.com/paper/1703.03020/) (2017)
- [InceptionGCN: Receptive Field Aware Graph Convolutional Network for Disease Prediction](https://mlsanity.com/paper/1903.04233/) (2019)
- [Deriving reproducible biomarkers from multi-site resting-state data: An Autism-based example](https://mlsanity.com/paper/1611.06066/) (2016)
- [Edge-variational Graph Convolutional Networks for Uncertainty-aware Disease Prediction](https://mlsanity.com/paper/2009.02759/) (2020)
- [RA-GCN: Graph Convolutional Network for Disease Prediction Problems with Imbalanced Data](https://mlsanity.com/paper/2103.00221/) (2021)
- [Zoom in to where it matters: a hierarchical graph based model for mammogram analysis](https://mlsanity.com/paper/1912.07517/) (2019)

## Built on by (most cited first, 3 of 3)

- [Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting it into MLPs: An Effective GNN-to-MLP Distillation Framework](https://mlsanity.com/paper/2305.10758/) (2023)
- [Graph-based Knowledge Distillation: A survey and experimental evaluation](https://mlsanity.com/paper/2302.14643/) (2023)
- [SCGG: A Deep Structure-Conditioned Graph Generative Model](https://mlsanity.com/paper/2209.09681/) (2022)

## Abstract (arXiv)

The increased amount of multi-modal medical data has opened the opportunities to simultaneously process various modalities such as imaging and non-imaging data to gain a comprehensive insight into the disease prediction domain. Recent studies using Graph Convolutional Networks (GCNs) provide novel semi-supervised approaches for integrating heterogeneous modalities while investigating the patients' associations for disease prediction. However, when the meta-data used for graph construction is not available at inference time (e.g., coming from a distinct population), the conventional methods exhibit poor performance. To address this issue, we propose a novel semi-supervised approach named GKD based on knowledge distillation. We train a teacher component that employs the label-propagation algorithm besides a deep neural network to benefit from the graph and non-graph modalities only in the training phase. The teacher component embeds all the available information into the soft pseudo-labels. The soft pseudo-labels are then used to train a deep student network for disease prediction of unseen test data for which the graph modality is unavailable. We perform our experiments on two public datasets for diagnosing Autism spectrum disorder, and Alzheimer's disease, along with a thorough analysis on synthetic multi-modal datasets. According to these experiments, GKD outperforms the previous graph-based deep learning methods in terms of accuracy, AUC, and Macro F1.