# FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning

arXiv: https://arxiv.org/abs/2404.14061 · HTML: https://mlsanity.com/paper/2404.14061/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Graph neural network distillation, one of 12 directions in Knowledge distillation.
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 22 Apr 2024.

## Placed in

- Knowledge distillation / [Graph neural network distillation](https://mlsanity.com/distill/graph-neural-network-distillation/)

## Builds on (most cited first, 27 of 27)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion](https://mlsanity.com/paper/1912.08795/) (2019)
- [Data-Free Knowledge Distillation for Heterogeneous Federated Learning](https://mlsanity.com/paper/2105.10056/) (2021)
- [Data-Free Learning of Student Networks](https://mlsanity.com/paper/1904.01186/) (2019)
- [FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction](https://mlsanity.com/paper/2203.11751/) (2022)
- [Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning](https://mlsanity.com/paper/2203.09249/) (2022)
- [Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis](https://mlsanity.com/paper/1903.05628/) (2019)
- [Subgraph Federated Learning with Missing Neighbor Generation](https://mlsanity.com/paper/2106.13430/) (2021)
- [Zero-Shot Knowledge Distillation in Deep Networks](https://mlsanity.com/paper/1905.08114/) (2019)
- [Federated Graph Classification over Non-IID Graphs](https://mlsanity.com/paper/2106.13423/) (2021)
- [Federated Learning on Non-IID Graphs via Structural Knowledge Sharing](https://mlsanity.com/paper/2211.13009/) (2022)
- [FedGTA: Topology-aware Averaging for Federated Graph Learning](https://mlsanity.com/paper/2401.11755/) (2024)
- [Semi-supervised Learning on Graphs with Generative Adversarial Nets](https://mlsanity.com/paper/1809.00130/) (2018)
- [Breaking the Entanglement of Homophily and Heterophily in Semi-supervised Node Classification](https://mlsanity.com/paper/2312.04111/) (2023)

## Built on by (most cited first, 27 of 27)

- [The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challenges](https://mlsanity.com/paper/2407.09618/) (2024)
- [Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglement](https://mlsanity.com/paper/2505.12684/) (2025)
- [S2FGL: Spatial Spectral Federated Graph Learning](https://mlsanity.com/paper/2507.02409/) (2025)
- [A Comprehensive Data-centric Overview of Federated Graph Learning](https://mlsanity.com/paper/2507.16541/) (2025)
- [Rethinking Federated Graph Learning: A Data Condensation Perspective](https://mlsanity.com/paper/2505.02573/) (2025)
- [Homophily Heterogeneity Matters in Graph Federated Learning: A Spectrum Sharing and Complementing Perspective](https://mlsanity.com/paper/2502.13732/) (2025)
- [Personalized One-shot Federated Graph Learning for Heterogeneous Clients](https://mlsanity.com/paper/2411.11304/) (2024)
- [TMTE: Effective Multimodal Graph Learning with Task-aware Modality and Topology Co-evolution](https://mlsanity.com/paper/2603.27723/) (2026)
- [STAGE: Tackling Semantic Drift in Multimodal Federated Graph Learning](https://mlsanity.com/paper/2605.11919/) (2026)
- [FedRGL: Robust Federated Graph Learning for Label Noise](https://mlsanity.com/paper/2411.18905/) (2024)
- [Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments](https://mlsanity.com/paper/2505.19699/) (2025)
- [Rethinking Federated Graph Foundation Models: A Graph-Language Alignment-based Approach](https://mlsanity.com/paper/2601.21369/) (2026)
- [Federated Prototype Graph Learning](https://mlsanity.com/paper/2504.09493/) (2025)
- [Towards Unbiased Federated Graph Learning: Label and Topology Perspectives](https://mlsanity.com/paper/2504.09963/) (2025)
- [Heterogeneity-Aware Knowledge Sharing for Graph Federated Learning](https://mlsanity.com/paper/2601.21589/) (2026)
- [Causality-inspired Federated Learning for Dynamic Spatio-Temporal Graphs](https://mlsanity.com/paper/2603.29384/) (2026)
- [Toward Federated Multimodal Graph Foundation Models: A Topology-Aware Multimodal Alignment Framework](https://mlsanity.com/paper/2607.15687/) (2026)
- [Towards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneity](https://mlsanity.com/paper/2608.00623/) (2026)
- [Adversarial Curriculum Graph-Free Knowledge Distillation for Graph Neural Networks](https://mlsanity.com/paper/2504.00540/) (2025)
- [FedSA-GCL: A Semi-Asynchronous Federated Graph Learning Framework with Personalized Aggregation and Cluster-Aware Broadcasting](https://mlsanity.com/paper/2507.18219/) (2025)
- [DFed-SST: Building Semantic- and Structure-aware Topologies for Decentralized Federated Graph Learning](https://mlsanity.com/paper/2508.11530/) (2025)
- [BoostFGL: Boosting Fairness in Federated Graph Learning](https://mlsanity.com/paper/2601.16496/) (2026)
- [Beyond Rigid Alignment: Graph Federated Learning via Dual Manifold Calibration](https://mlsanity.com/paper/2605.06260/) (2026)
- [Generalized Category Discovery in Federated Graph Learning](https://mlsanity.com/paper/2605.08178/) (2026)
- [UMEDA: Unified Multi-modal Efficient Data Fusion for Privacy-Preserving Graph Federated Learning via Spectral-Gated Attention and Diffusion-Based Operator Alignment](https://mlsanity.com/paper/2605.08288/) (2026)
- [Towards Federated Long-Tailed Graph Learning: An Energy-Guided Dual Decoupling Approach](https://mlsanity.com/paper/2606.24237/) (2026)
- [Towards Adaptive Federated Graph Clustering: A Global Community-aware Contrastive Learning-based Approach](https://mlsanity.com/paper/2609.26063/) (2026)

## Abstract (arXiv)

Subgraph federated learning (subgraph-FL) is a new distributed paradigm that facilitates the collaborative training of graph neural networks (GNNs) by multi-client subgraphs. Unfortunately, a significant challenge of subgraph-FL arises from subgraph heterogeneity, which stems from node and topology variation, causing the impaired performance of the global GNN. Despite various studies, they have not yet thoroughly investigated the impact mechanism of subgraph heterogeneity. To this end, we decouple node and topology variation, revealing that they correspond to differences in label distribution and structure homophily. Remarkably, these variations lead to significant differences in the class-wise knowledge reliability of multiple local GNNs, misguiding the model aggregation with varying degrees. Building on this insight, we propose topology-aware data-free knowledge distillation technology (FedTAD), enhancing reliable knowledge transfer from the local model to the global model. Extensive experiments on six public datasets consistently demonstrate the superiority of FedTAD over state-of-the-art baselines.