mlsanityData to 7 Oct 2026

Personalized Decentralized Federated Learning with Knowledge Distillation

arXiv 2302.12156 ↗First posted 23 Feb 2023cs.LG cs.IT eess.SPcited by 21

Not on the map. Why?

Abstract (arXiv)

Personalization in federated learning (FL) functions as a coordinator for clients with high variance in data or behavior. Ensuring the convergence of these clients' models relies on how closely users collaborate with those with similar patterns or preferences. However, it is generally challenging to quantify similarity under limited knowledge about other users' models given to users in a decentralized network. To cope with this issue, we propose a personalized and fully decentralized FL algorithm, leveraging knowledge distillation techniques to empower each device so as to discern statistical distances between local models. Each client device can enhance its performance without sharing local data by estimating the similarity between two intermediate outputs from feeding local samples as in knowledge distillation. Our empirical studies demonstrate that the proposed algorithm improves the test accuracy of clients in fewer iterations under highly non-independent and identically distributed (non-i.i.d.) data distributions and is beneficial to agents with small datasets, even without the need for a central server.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 24 of its 30 references that are in the map. First posted on arXiv 23 Feb 2023.

Builds on

24 of 30 references.

References · 24

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  5. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  6. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  7. 2017Federated Multi-Task Learningcited by 669
  8. 2020Federated Learning with Matched Averagingcited by 564
  9. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  10. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  11. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  12. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
12 more
  1. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  2. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  3. 2018Large scale distributed neural network training through online distillationcited by 144
  4. 2019Federated Learning for Wireless Communications: Motivation, Opportunities and Challengescited by 128
  5. 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 73
  6. 2021Personalized Federated Learning through Local Memorizationcited by 57
  7. 2017Personalized and Private Peer-to-Peer Machine Learningcited by 54
  8. 2019Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphscited by 34
  9. 2022Motley: Benchmarking Heterogeneity and Personalization in Federated Learningcited by 31
  10. 2021Decentralized Personalized Federated Learning: Lower Bounds and Optimal Algorithm for All Personalization Modescited by 13
  11. 2021Decentralized Personalized Federated Learning for Min-Max Problemscited by 4
  12. 2021Unifying Distillation with Personalization in Federated Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 6; 2026: 5.

21 papers.

Sort:

  1. 2024Green Federated Learning: A new era of Green Aware AIcited by 12
  2. 2024Decentralized Directed Collaboration for Personalized Federated Learningcited by 9
  3. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  4. 2024Boosting Asynchronous Decentralized Learning with Model Fragmentationcited by 6
  5. 2024DRACO: Decentralized Asynchronous Federated Learning over Row-Stochastic Wireless Networkscited by 5
  6. 2024FedSPD: A Soft-clustering Approach for Personalized Decentralized Federated Learningcited by 4
  7. 2024Smart Sampling: Helping from Friendly Neighbors for Decentralized Federated Learningcited by 2
  8. 2026Learning to Collaborate: An Orchestrated-Decentralized Framework for Peer-to-Peer LLM Federationcited by 2
  9. 2026CRAD: Class-wise Reliability-Aware Distillation for Decentralized Heterogeneous Federated Learningcited by 1
  10. 2024MAPL: Model Agnostic Peer-to-peer Learningcited by 0
  11. 2024P4: Towards private, personalized, and Peer-to-Peer learningcited by 0
  12. 2024Trust-free Personalized Decentralized Learningcited by 0
9 more
  1. 2025A Novel Algorithm for Personalized Federated Learning: Knowledge Distillation with Weighted Combination Losscited by 0
  2. 2025Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learningcited by 0
  3. 2025Client Clustering Meets Knowledge Sharing: Enhancing Privacy and Robustness in Personalized Peer-to-Peer Learningcited by 0
  4. 2025Whom to Trust? Adaptive Collaboration in Personalized Federated Learningcited by 0
  5. 2025Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learningcited by 0
  6. 2025Feature-Based Semantics-Aware Scheduling for Energy-Harvesting Federated Learningcited by 0
  7. 2026CALM: Class-wise Agreement and Label-gated Disagreement Modulation for Decentralized Federated Learningcited by 0
  8. 2026Robust Decentralized Federated Distillation via Multi-Modality Knowledge Collaborationcited by 0
  9. 2026Robust Decentralized Personalized Federated Learning via Prediction-Constrained Neighborhood Collaborationcited by 0