# Decentralized federated learning of deep neural networks on non-iid data arXiv: https://arxiv.org/abs/2107.08517 ยท HTML: https://mlsanity.com/paper/2107.08517/ ## Key facts - Cited by 25 papers in the map (data through 14 Oct 2026). - Builds on 16 of its 24 references that are in the map. - First posted on arXiv 18 Jul 2021. ## Builds on (most cited first, 16 of 16) - [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016) - [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017) - [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019) - [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018) - [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020) - [Adaptive Personalized Federated Learning](https://mlsanity.com/paper/2003.13461/) (2020) - [Improving Federated Learning Personalization via Model Agnostic Meta Learning](https://mlsanity.com/paper/1909.12488/) (2019) - [Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data](https://mlsanity.com/paper/1811.11479/) (2018) - [Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication](https://mlsanity.com/paper/1902.00340/) (2019) - [Stochastic Gradient Push for Distributed Deep Learning](https://mlsanity.com/paper/1811.10792/) (2018) - [Federated Evaluation of On-device Personalization](https://mlsanity.com/paper/1910.10252/) (2019) - [BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning](https://mlsanity.com/paper/1905.06731/) (2019) - [MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling](https://mlsanity.com/paper/1905.09435/) (2019) - [Consensus Control for Decentralized Deep Learning](https://mlsanity.com/paper/2102.04828/) (2021) - [Gossip training for deep learning](https://mlsanity.com/paper/1611.09726/) (2016) - [Specialized federated learning using a mixture of experts](https://mlsanity.com/paper/2010.02056/) (2020) ## Built on by (most cited first, 25 of 25) - [Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges](https://mlsanity.com/paper/2211.08413/) (2022) - [A Survey on Decentralized Federated Learning](https://mlsanity.com/paper/2308.04604/) (2023) - [Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matching](https://mlsanity.com/paper/2203.12285/) (2022) - [Epidemic Learning: Boosting Decentralized Learning with Randomized Communication](https://mlsanity.com/paper/2310.01972/) (2023) - [Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directions](https://mlsanity.com/paper/2411.12377/) (2024) - [Decentralized adaptive clustering of deep nets is beneficial for client collaboration](https://mlsanity.com/paper/2206.08839/) (2022) - [Efficient Node Selection in Private Personalized Decentralized Learning](https://mlsanity.com/paper/2301.12755/) (2023) - [Towards Heterogeneity-Aware and Energy-Efficient Topology Optimization for Decentralized Federated Learning in Edge Environment](https://mlsanity.com/paper/2508.08278/) (2025) - [Smart Sampling: Helping from Friendly Neighbors for Decentralized Federated Learning](https://mlsanity.com/paper/2407.04460/) (2024) - [pFedGame -- Decentralized Federated Learning using Game Theory in Dynamic Topology](https://mlsanity.com/paper/2410.04058/) (2024) - [Communication-Efficient Diffusion Strategy for Performance Improvement of Federated Learning with Non-IID Data](https://mlsanity.com/paper/2207.07493/) (2022) - [FedSR: A Semi-Decentralized Federated Learning Algorithm for Non-IIDness in IoT System](https://mlsanity.com/paper/2403.14718/) (2024) - [On the effects of similarity metrics in decentralized deep learning under distributional shift](https://mlsanity.com/paper/2409.10720/) (2024) - [Fair Decentralized Learning](https://mlsanity.com/paper/2410.02541/) (2024) - [FedPAE: Peer-Adaptive Ensemble Learning for Asynchronous and Model-Heterogeneous Federated Learning](https://mlsanity.com/paper/2410.14075/) (2024) - [Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization](https://mlsanity.com/paper/2412.02535/) (2024) - [Concept-aware clustering for decentralized deep learning under temporal shift](https://mlsanity.com/paper/2306.12768/) (2023) - [Adaptive Decentralized Federated Learning in Energy and Latency Constrained Wireless Networks](https://mlsanity.com/paper/2403.20075/) (2024) - [DFed-SST: Building Semantic- and Structure-aware Topologies for Decentralized Federated Graph Learning](https://mlsanity.com/paper/2508.11530/) (2025) - [Distributed Gossip-GAN for Low-overhead CSI Feedback Training in FDD mMIMO-OFDM Systems](https://mlsanity.com/paper/2509.10490/) (2025) - [DFCA: Decentralized Federated Clustering Algorithm](https://mlsanity.com/paper/2510.15300/) (2025) - [Function-Space ADMM for Decentralized Federated Learning: A Control Theoretic Perspective](https://mlsanity.com/paper/2605.09356/) (2026) - [SAGE: Optimal-Stopping Peer Selection for Decentralised Federated Learning](https://mlsanity.com/paper/2609.23773/) (2026) - [Population Scaling or Data Dilution? Dynamics of Local Topology Evolution in Decentralized Learning](https://mlsanity.com/paper/2610.05476/) (2026) - [Tram-FL: Reducing Communication and Computation Costs through Sequential Model Circulation in Decentralized Federated Learning](https://mlsanity.com/paper/2610.07859/) (2026) ## Abstract (arXiv) We tackle the non-convex problem of learning a personalized deep learning model in a decentralized setting. More specifically, we study decentralized federated learning, a peer-to-peer setting where data is distributed among many clients and where there is no central server to orchestrate the training. In real world scenarios, the data distributions are often heterogeneous between clients. Therefore, in this work we study the problem of how to efficiently learn a model in a peer-to-peer system with non-iid client data. We propose a method named Performance-Based Neighbor Selection (PENS) where clients with similar data distributions detect each other and cooperate by evaluating their training losses on each other's data to learn a model suitable for the local data distribution. Our experiments on benchmark datasets show that our proposed method is able to achieve higher accuracies as compared to strong baselines.