mlsanityData to 14 Oct 2026

Decentralized federated learning of deep neural networks on non-iid data

arXiv 2107.08517 ↗First posted 18 Jul 2021cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

We tackle the non-convex problem of learning a personalized deep learning model in a decentralized setting. More specifically, we study decentralized federated learning, a peer-to-peer setting where data is distributed among many clients and where there is no central server to orchestrate the training. In real world scenarios, the data distributions are often heterogeneous between clients. Therefore, in this work we study the problem of how to efficiently learn a model in a peer-to-peer system with non-iid client data. We propose a method named Performance-Based Neighbor Selection (PENS) where clients with similar data distributions detect each other and cooperate by evaluating their training losses on each other's data to learn a model suitable for the local data distribution. Our experiments on benchmark datasets show that our proposed method is able to achieve higher accuracies as compared to strong baselines.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 16 of its 24 references that are in the map. First posted on arXiv 18 Jul 2021.

Builds on

16 of 24 references.

References · 16

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2018Federated Learning with Non-IID Datacited by 1,108
  5. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  6. 2020Adaptive Personalized Federated Learningcited by 358
  7. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  8. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
  9. 2019Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communicationcited by 221
  10. 2018Stochastic Gradient Push for Distributed Deep Learningcited by 152
  11. 2019Federated Evaluation of On-device Personalizationcited by 147
  12. 2019BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learningcited by 117
4 more
  1. 2019MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Samplingcited by 67
  2. 2021Consensus Control for Decentralized Deep Learningcited by 49
  3. 2016Gossip training for deep learningcited by 35
  4. 2020Specialized federated learning using a mixture of expertscited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 4; 2024: 9; 2025: 4; 2026: 4.

25 papers.

Sort:

  1. 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 138
  2. 2023A Survey on Decentralized Federated Learningcited by 27
  3. 2022Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matchingcited by 24
  4. 2023Epidemic Learning: Boosting Decentralized Learning with Randomized Communicationcited by 21
  5. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 13
  6. 2022Decentralized adaptive clustering of deep nets is beneficial for client collaborationcited by 7
  7. 2023Efficient Node Selection in Private Personalized Decentralized Learningcited by 4
  8. 2025Towards Heterogeneity-Aware and Energy-Efficient Topology Optimization for Decentralized Federated Learning in Edge Environmentcited by 3
  9. 2024Smart Sampling: Helping from Friendly Neighbors for Decentralized Federated Learningcited by 2
  10. 2024pFedGame -- Decentralized Federated Learning using Game Theory in Dynamic Topologycited by 2
  11. 2022Communication-Efficient Diffusion Strategy for Performance Improvement of Federated Learning with Non-IID Datacited by 1
  12. 2024FedSR: A Semi-Decentralized Federated Learning Algorithm for Non-IIDness in IoT Systemcited by 1
13 more
  1. 2024On the effects of similarity metrics in decentralized deep learning under distributional shiftcited by 1
  2. 2024Fair Decentralized Learningcited by 1
  3. 2024FedPAE: Peer-Adaptive Ensemble Learning for Asynchronous and Model-Heterogeneous Federated Learningcited by 1
  4. 2024Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimizationcited by 1
  5. 2023Concept-aware clustering for decentralized deep learning under temporal shiftcited by 0
  6. 2024Adaptive Decentralized Federated Learning in Energy and Latency Constrained Wireless Networkscited by 0
  7. 2025DFed-SST: Building Semantic- and Structure-aware Topologies for Decentralized Federated Graph Learningcited by 0
  8. 2025Distributed Gossip-GAN for Low-overhead CSI Feedback Training in FDD mMIMO-OFDM Systemscited by 0
  9. 2025DFCA: Decentralized Federated Clustering Algorithmcited by 0
  10. 2026Function-Space ADMM for Decentralized Federated Learning: A Control Theoretic Perspectivecited by 0
  11. 2026SAGE: Optimal-Stopping Peer Selection for Decentralised Federated Learningcited by 0
  12. 2026Population Scaling or Data Dilution? Dynamics of Local Topology Evolution in Decentralized Learningcited by 0
  13. 2026Tram-FL: Reducing Communication and Computation Costs through Sequential Model Circulation in Decentralized Federated Learningcited by 0