mlsanityData to 14 Oct 2026

FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning

arXiv 2404.14061 ↗First posted 22 Apr 2024cs.LG cs.AI cs.DBcited by 27

On the map: Knowledge distillation → Graph neural network distillation

Abstract (arXiv)

Subgraph federated learning (subgraph-FL) is a new distributed paradigm that facilitates the collaborative training of graph neural networks (GNNs) by multi-client subgraphs. Unfortunately, a significant challenge of subgraph-FL arises from subgraph heterogeneity, which stems from node and topology variation, causing the impaired performance of the global GNN. Despite various studies, they have not yet thoroughly investigated the impact mechanism of subgraph heterogeneity. To this end, we decouple node and topology variation, revealing that they correspond to differences in label distribution and structure homophily. Remarkably, these variations lead to significant differences in the class-wise knowledge reliability of multiple local GNNs, misguiding the model aggregation with varying degrees. Building on this insight, we propose topology-aware data-free knowledge distillation technology (FedTAD), enhancing reliable knowledge transfer from the local model to the global model. Extensive experiments on six public datasets consistently demonstrate the superiority of FedTAD over state-of-the-art baselines.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Placed in Graph neural network distillation, one of 12 directions in Knowledge distillation. Builds on 27 of its 35 references that are in the map. First posted on arXiv 22 Apr 2024.

Builds on

27 of 35 references.

References · 27

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  8. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  9. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  10. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  11. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,102
  12. 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
15 more
  1. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 976
  2. 2021Model-Contrastive Federated Learningcited by 766
  3. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  4. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  5. 2019Data-Free Learning of Student Networkscited by 204
  6. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  7. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  8. 2019Mode Seeking Generative Adversarial Networks for Diverse Image Synthesiscited by 138
  9. 2021Subgraph Federated Learning with Missing Neighbor Generationcited by 132
  10. 2019Zero-Shot Knowledge Distillation in Deep Networkscited by 121
  11. 2021Federated Graph Classification over Non-IID Graphscited by 114
  12. 2022Federated Learning on Non-IID Graphs via Structural Knowledge Sharingcited by 88
  13. 2024FedGTA: Topology-aware Averaging for Federated Graph Learningcited by 44
  14. 2018Semi-supervised Learning on Graphs with Generative Adversarial Netscited by 21
  15. 2023Breaking the Entanglement of Homophily and Heterophily in Semi-supervised Node Classificationcited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 11; 2026: 13.

27 papers.

Sort:

By area:

  1. 2024The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challengescited by 29
  2. 2025Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglementcited by 14
  3. 2025S2FGL: Spatial Spectral Federated Graph Learningcited by 7
  4. 2025A Comprehensive Data-centric Overview of Federated Graph Learningcited by 7
  5. 2025Rethinking Federated Graph Learning: A Data Condensation Perspectivecited by 6
  6. 2025Homophily Heterogeneity Matters in Graph Federated Learning: A Spectrum Sharing and Complementing Perspectivecited by 4
  7. 2024Personalized One-shot Federated Graph Learning for Heterogeneous Clientscited by 3
  8. 2026TMTE: Effective Multimodal Graph Learning with Task-aware Modality and Topology Co-evolutioncited by 3
  9. 2026STAGE: Tackling Semantic Drift in Multimodal Federated Graph Learningcited by 3
  10. 2024FedRGL: Robust Federated Graph Learning for Label Noisecited by 2
  11. 2025Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environmentscited by 2
  12. 2026Rethinking Federated Graph Foundation Models: A Graph-Language Alignment-based Approachcited by 2
15 more
  1. 2025Federated Prototype Graph Learningcited by 1
  2. 2025Towards Unbiased Federated Graph Learning: Label and Topology Perspectivescited by 1
  3. 2026Heterogeneity-Aware Knowledge Sharing for Graph Federated Learningcited by 1
  4. 2026Causality-inspired Federated Learning for Dynamic Spatio-Temporal Graphscited by 1
  5. 2026Toward Federated Multimodal Graph Foundation Models: A Topology-Aware Multimodal Alignment Frameworkcited by 1
  6. 2026Towards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneitycited by 1
  7. 2025Adversarial Curriculum Graph-Free Knowledge Distillation for Graph Neural Networkscited by 0
  8. 2025FedSA-GCL: A Semi-Asynchronous Federated Graph Learning Framework with Personalized Aggregation and Cluster-Aware Broadcastingcited by 0
  9. 2025DFed-SST: Building Semantic- and Structure-aware Topologies for Decentralized Federated Graph Learningcited by 0
  10. 2026BoostFGL: Boosting Fairness in Federated Graph Learningcited by 0
  11. 2026Beyond Rigid Alignment: Graph Federated Learning via Dual Manifold Calibrationcited by 0
  12. 2026Generalized Category Discovery in Federated Graph Learningcited by 0
  13. 2026UMEDA: Unified Multi-modal Efficient Data Fusion for Privacy-Preserving Graph Federated Learning via Spectral-Gated Attention and Diffusion-Based Operator Alignmentcited by 0
  14. 2026Towards Federated Long-Tailed Graph Learning: An Energy-Guided Dual Decoupling Approachcited by 0
  15. 2026Towards Adaptive Federated Graph Clustering: A Global Community-aware Contrastive Learning-based Approachcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Knowledge-Driven Federated Graph Learning on Model Heterogeneity
  2. 2023Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearning
  3. 2024A Teacher-Free Graph Knowledge Distillation Framework with Dual Self-Distillation
  4. 2022Geometric Knowledge Distillation: Topology Compression for Graph Neural Networks
  5. 2021On Representation Knowledge Distillation for Graph Neural Networks
  6. 2024Federated Graph Learning with Graphless Clients
  7. 2026Structural Entropy-Driven Graph Diffusion Generation for One-Shot Federated Graph Learning
  8. 2026Communication-efficient Federated Graph Classification via Generative Diffusion Modeling