mlsanityData to 14 Oct 2026

FedBKD: Distilled Federated Learning to Embrace Gerneralization and Personalization on Non-IID Data

arXiv 2506.20245 ↗First posted 25 Jun 2025cs.LG cs.AI cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a decentralized collaborative machine learning (ML) technique. It provides a solution to the issues of isolated data islands and data privacy leakage in industrial ML practices. One major challenge in FL is handling the non-identical and independent distributed (non-IID) data. Current solutions either focus on constructing an all-powerful global model, or customizing personalized local models. Few of them can provide both a well-generalized global model and well-performed local models at the same time. Additionally, many FL solutions to the non-IID problem are benefited from introducing public datasets. However, this will also increase the risk of data leakage. To tackle the problems, we propose a novel data-free distillation framework, Federated Bidirectional Knowledge Distillation (FedBKD). Specifically, we train Generative Adversarial Networks (GAN) for synthetic data. During the GAN training, local models serve as discriminators and their parameters are frozen. The synthetic data is then used for bidirectional distillation between global and local models to achieve knowledge interactions so that performances for both sides are improved. We conduct extensive experiments on 4 benchmarks under different non-IID settings. The results show that FedBKD achieves SOTA performances in every case.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 45 references that are in the map. First posted on arXiv 25 Jun 2025.

Builds on

36 of 45 references.

References · 36

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  3. 2018LEAF: A Benchmark for Federated Settingscited by 792
  4. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  5. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  6. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  7. 2020Federated Learning with Matched Averagingcited by 564
  8. 2019Federated Learning with Personalization Layerscited by 541
  9. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  10. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  11. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  12. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
24 more
  1. 2020Adaptive Personalized Federated Learningcited by 358
  2. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  3. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
  4. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
  5. 2021Personalized Federated Learning using Hypernetworkscited by 212
  6. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  7. 2019Data-Free Learning of Student Networkscited by 204
  8. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  9. 2020Survey of Personalization Techniques for Federated Learningcited by 145
  10. 2021Federated Learning for Open Bankingcited by 141
  11. 2019Mode Seeking Generative Adversarial Networks for Diverse Image Synthesiscited by 138
  12. 2019One-Shot Federated Learningcited by 125
  13. 2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
  14. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  15. 2021Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learningcited by 53
  16. 2021FedCG: Leverage Conditional GAN for Protecting Privacy and Maintaining Competitive Performance in Federated Learningcited by 39
  17. 2022Personalized Federated Learning With Graphcited by 29
  18. 2021Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneitycited by 27
  19. 2021Personalized Federated Learning with Contextualized Generalizationcited by 23
  20. 2021FedMatch: Federated Learning Over Heterogeneous Question Answering Datacited by 23
  21. 2021On Bridging Generic and Personalized Federated Learning for Image Classificationcited by 15
  22. 2022Meta-Learning Based Knowledge Extrapolation for Knowledge Graphs in the Federated Settingcited by 15
  23. 2022FedDKD: Federated Learning with Decentralized Knowledge Distillationcited by 9
  24. 2021Federated Learning for Privacy-Preserving Open Innovation Future on Digital Healthcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Secure, Verifiable, and Scalable Multi-Client Data Sharing via Consensus-Based Privacy-Preserving Data Distributioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022FedDTG:Federated Data-Free Knowledge Distillation via Three-Player Generative Adversarial Networks
  2. 2024Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutions
  3. 2024Federated Distillation: A Survey
  4. 2024FedSiKD: Clients Similarity and Knowledge Distillation: Addressing Non-i.i.d. and Constraints in Federated Learning
  5. 2023FedAL: Black-Box Federated Knowledge Distillation Enabled by Adversarial Learning
  6. 2022Knowledge Distillation for Federated Learning: a Practical Guide
  7. 2024FedD2S: Personalized Data-Free Federated Knowledge Distillation
  8. 2023The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillation