mlsanityData to 14 Oct 2026

Decentralized Federated Learning via Mutual Knowledge Transfer

arXiv 2012.13063 ↗First posted 24 Dec 2020cs.LG cs.DCcited by 43

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate the problem of decentralized federated learning (DFL) in Internet of things (IoT) systems, where a number of IoT clients train models collectively for a common task without sharing their private training data in the absence of a central server. Most of the existing DFL schemes are composed of two alternating steps, i.e., model updating and model averaging. However, averaging model parameters directly to fuse different models at the local clients suffers from client-drift especially when the training data are heterogeneous across different clients. This leads to slow convergence and degraded learning performance. As a possible solution, we propose the decentralized federated earning via mutual knowledge transfer (Def-KT) algorithm where local clients fuse models by transferring their learnt knowledge to each other. Our experiments on the MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets reveal that the proposed Def-KT algorithm significantly outperforms the baseline DFL methods with model averaging, i.e., Combo and FullAvg, especially when the training data are not independent and identically distributed (non-IID) across different clients.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 9; 2023: 10; 2024: 10; 2025: 5; 2026: 5.

43 papers.

Sort:

By area:

  1. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  2. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  3. 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 138
  4. 2021Decentralized Federated Learning through Proxy Model Sharingcited by 57
  5. 2021Deep Federated Learning for Autonomous Drivingcited by 54
  6. 2022A Survey on Heterogeneous Federated Learningcited by 42
  7. 2023A Survey on Decentralized Federated Learningcited by 27
  8. 2022Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matchingcited by 24
  9. 2024Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchorscited by 21
  10. 2022Find Your Friends: Personalized Federated Learning with the Right Collaboratorscited by 10
  11. 2024Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutionscited by 9
  12. 2023Advancements in Federated Learning: Models, Methods, and Privacycited by 7
31 more
  1. 2023Knowledge Distillation in Federated Edge Learning: A Surveycited by 5
  2. 2022PEPPER: Empowering User-Centric Recommender Systems over Gossip Learningcited by 4
  3. 2024DFML: Decentralized Federated Mutual Learningcited by 4
  4. 2021DACFL: Dynamic Average Consensus Based Federated Learning in Decentralized Topologycited by 3
  5. 2023Homogenizing Non-IID datasets via In-Distribution Knowledge Distillation for Decentralized Learningcited by 3
  6. 2023Sparse Decentralized Federated Learningcited by 2
  7. 2023Federated Learning with Neural Graphical Modelscited by 2
  8. 2024Practical Insights into Knowledge Distillation for Pre-Trained Modelscited by 2
  9. 2025PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distributioncited by 2
  10. 2022Nonparametric Decentralized Detection and Sparse Sensor Selection via Multi-Sensor Online Kernel Scalar Quantizationcited by 1
  11. 2022Reducing Training Time in Cross-Silo Federated Learning using Multigraph Topologycited by 1
  12. 2023Collaborative Distributed Machine Learningcited by 1
  13. 2024A review on different techniques used to combat the non-IID and heterogeneous nature of data in FLcited by 1
  14. 2024Gradient Coding in Decentralized Learning for Evading Stragglerscited by 1
  15. 2024FedSR: A Semi-Decentralized Federated Learning Algorithm for Non-IIDness in IoT Systemcited by 1
  16. 2024Adaptive Coded Federated Learning: Privacy Preservation and Straggler Mitigationcited by 1
  17. 2024Exposing the Vulnerability of Decentralized Learning to Membership Inference Attacks Through the Lens of Graph Mixingcited by 1
  18. 2025Federated prediction for scalable and privacy-preserved knowledge-based planning in radiotherapycited by 1
  19. 2025Distribution-Aware Mobility-Assisted Decentralized Federated Learningcited by 1
  20. 2022FedSiam-DA: Dual-aggregated Federated Learning via Siamese Network under Non-IID Datacited by 0
  21. 2022FLAGS Framework for Comparative Analysis of Federated Learning Algorithmscited by 0
  22. 2023Private Networked Federated Learning for Nonsmooth Objectivescited by 0
  23. 2023Cross-feature Contrastive Loss for Decentralized Deep Learning on Heterogeneous Datacited by 0
  24. 2024Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selectioncited by 0
  25. 2025Privacy Preserving and Robust Aggregation for Cross-Silo Federated Learning in Non-IID Settingscited by 0
  26. 2025FedOAED: Federated On-Device Autoencoder Denoiser for Heterogeneous Data under Limited Client Availabilitycited by 0
  27. 2026FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participationcited by 0
  28. 2026DECA: Decentralizing Block-Wise Adam for Efficient LLM Full-Parameter Fine-Tuning on Non-IID Datacited by 0
  29. 2026DPDL: Towards Differential Privacy Preservation in Decentralized Stochastic Learning on Non-IID Datacited by 0
  30. 2026Robust Decentralized Personalized Federated Learning via Prediction-Constrained Neighborhood Collaborationcited by 0
  31. 2026PROSE: A Theory of Optimal Stopping with Perishable Evidence for Peer Selection in Intermittently Connected Decentralised Learningcited by 0