mlsanityData to 7 Oct 2026

A Novel Algorithm for Personalized Federated Learning: Knowledge Distillation with Weighted Combination Loss

arXiv 2504.04642 ↗First posted 6 Apr 2025stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) offers a privacy-preserving framework for distributed machine learning, enabling collaborative model training across diverse clients without centralizing sensitive data. However, statistical heterogeneity, characterized by non-independent and identically distributed (non-IID) client data, poses significant challenges, leading to model drift and poor generalization. This paper proposes a novel algorithm, pFedKD-WCL (Personalized Federated Knowledge Distillation with Weighted Combination Loss), which integrates knowledge distillation with bi-level optimization to address non-IID challenges. pFedKD-WCL leverages the current global model as a teacher to guide local models, optimizing both global convergence and local personalization efficiently. We evaluate pFedKD-WCL on the MNIST dataset and a synthetic dataset with non-IID partitioning, using multinomial logistic regression and multilayer perceptron models. Experimental results demonstrate that pFedKD-WCL outperforms state-of-the-art algorithms, including FedAvg, FedProx, Per-FedAvg, and pFedMe, in terms of accuracy and convergence speed.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 6 Apr 2025.

Builds on

20 of 27 references.

References · 20

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2018Federated Learning with Non-IID Datacited by 1,099
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  7. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  9. 2017Federated Multi-Task Learningcited by 669
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  11. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  12. 2019Federated Learning with Personalization Layerscited by 533
8 more
  1. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  2. 2021Towards Personalized Federated Learningcited by 462
  3. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  4. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  5. 2020Adaptive Personalized Federated Learningcited by 353
  6. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  7. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  8. 2023Personalized Decentralized Federated Learning with Knowledge Distillationcited by 21

Built on by

Not cited yet.