mlsanityData to 7 Oct 2026

Dynamic Topology Optimization for Non-IID Data in Decentralized Learning

arXiv 2602.03383 ↗First posted 3 Feb 2026cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Decentralized learning (DL) enables a set of nodes to train a model collaboratively without central coordination, offering benefits for privacy and scalability. However, DL struggles to train a high accuracy model when the data distribution is non-independent and identically distributed (non-IID) and when the communication topology is static. To address these issues, we propose Morph, a topology optimization algorithm for DL. In Morph, nodes adaptively choose peers for model exchange based on maximum model dissimilarity. Morph maintains a fixed in-degree while dynamically reshaping the communication graph through gossip-based peer discovery and diversity-driven neighbor selection, thereby improving robustness to data heterogeneity. Experiments on CIFAR-10 and FEMNIST with up to 100 nodes show that Morph consistently outperforms static and epidemic baselines, while closely tracking the fully connected upper bound. On CIFAR-10, Morph achieves a relative improvement of 1.12x in test accuracy compared to the state-of-the-art baselines. On FEMNIST, Morph achieves an accuracy that is 1.08x higher than Epidemic Learning. Similar trends hold for 50 node deployments, where Morph narrows the gap to the fully connected upper bound within 0.5 percentage points on CIFAR-10. These results demonstrate that Morph achieves higher final accuracy, faster convergence, and more stable learning as quantified by lower inter-node variance, while requiring fewer communication rounds than baselines and no global knowledge.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 54 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

33 of 54 references.

References · 33

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  4. 2018LEAF: A Benchmark for Federated Settingscited by 780
  5. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  6. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  7. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  8. 2018Stochastic Gradient Push for Distributed Deep Learningcited by 152
  9. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  10. 2021Exponential Graph is Provably Efficient for Decentralized Deep Trainingcited by 55
  11. 2021Consensus Control for Decentralized Deep Learningcited by 49
  12. 2021Cross-Gradient Aggregation for Decentralized Learning from Non-IID datacited by 32
21 more
  1. 2022Refined Convergence and Topology Learning for Decentralized SGD with Heterogeneous Datacited by 31
  2. 2022Communication-Efficient Topologies for Decentralized Learning with O(1) Consensus Ratecited by 25
  3. 2023Epidemic Learning: Boosting Decentralized Learning with Randomized Communicationcited by 21
  4. 2023Decentralized Learning Made Easy with DecentralizePycited by 18
  5. 2021BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learningcited by 16
  6. 2022AGIC: Approximate Gradient Inversion Attack on Federated Learningcited by 15
  7. 2024Training Diffusion Models with Federated Learningcited by 15
  8. 2021A Triangle Inequality for Cosine Similaritycited by 13
  9. 2022Data-heterogeneity-aware Mixing for Decentralized Learningcited by 12
  10. 2023Semantic-aware Node Synthesis for Imbalanced Heterogeneous Information Networkscited by 9
  11. 2021Decentralized Federated Learning Preserves Model and Data Privacycited by 8
  12. 2024Impact of network topology on the performance of Decentralized Federated Learningcited by 7
  13. 2022Aergia: Leveraging Heterogeneity in Federated Learning Systemscited by 6
  14. 2024Provable Privacy Advantages of Decentralized Federated Learning via Distributed Optimizationcited by 6
  15. 2022MUDGUARD: Taming Malicious Majorities in Federated Learning using Privacy-Preserving Byzantine-Robust Clusteringcited by 5
  16. 2022Neighborhood Gradient Clustering: An Efficient Decentralized Learning Method for Non-IID Data Distributionscited by 5
  17. 2024Asynchronous Multi-Server Federated Learning for Geo-Distributed Clientscited by 2
  18. 2024pFedGame -- Decentralized Federated Learning using Game Theory in Dynamic Topologycited by 2
  19. 2016Learnt quasi-transitive similarity for retrieval from large collections of facescited by 1
  20. 2024CCBNet: Confidential Collaborative Bayesian Networks Inferencecited by 1
  21. 2024Asynchronous Byzantine Federated Learningcited by 1

Built on by

Not cited yet.