# Dynamic Topology Optimization for Non-IID Data in Decentralized Learning

arXiv: https://arxiv.org/abs/2602.03383 · HTML: https://mlsanity.com/paper/2602.03383/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 54 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Builds on (most cited first, 30 of 33)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent](https://mlsanity.com/paper/1705.09056/) (2017)
- [A Unified Theory of Decentralized SGD with Changing Topology and Local Updates](https://mlsanity.com/paper/2003.10422/) (2020)
- [The Non-IID Data Quagmire of Decentralized Machine Learning](https://mlsanity.com/paper/1910.00189/) (2019)
- [Stochastic Gradient Push for Distributed Deep Learning](https://mlsanity.com/paper/1811.10792/) (2018)
- [FedScale: Benchmarking Model and System Performance of Federated Learning at Scale](https://mlsanity.com/paper/2105.11367/) (2021)
- [Exponential Graph is Provably Efficient for Decentralized Deep Training](https://mlsanity.com/paper/2110.13363/) (2021)
- [Consensus Control for Decentralized Deep Learning](https://mlsanity.com/paper/2102.04828/) (2021)
- [Cross-Gradient Aggregation for Decentralized Learning from Non-IID data](https://mlsanity.com/paper/2103.02051/) (2021)
- [Refined Convergence and Topology Learning for Decentralized SGD with Heterogeneous Data](https://mlsanity.com/paper/2204.04452/) (2022)
- [Communication-Efficient Topologies for Decentralized Learning with O(1) Consensus Rate](https://mlsanity.com/paper/2210.07881/) (2022)
- [Epidemic Learning: Boosting Decentralized Learning with Randomized Communication](https://mlsanity.com/paper/2310.01972/) (2023)
- [Decentralized Learning Made Easy with DecentralizePy](https://mlsanity.com/paper/2304.08322/) (2023)
- [BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learning](https://mlsanity.com/paper/2111.04287/) (2021)
- [AGIC: Approximate Gradient Inversion Attack on Federated Learning](https://mlsanity.com/paper/2204.13784/) (2022)
- [Training Diffusion Models with Federated Learning](https://mlsanity.com/paper/2406.12575/) (2024)
- [A Triangle Inequality for Cosine Similarity](https://mlsanity.com/paper/2107.04071/) (2021)
- [Data-heterogeneity-aware Mixing for Decentralized Learning](https://mlsanity.com/paper/2204.06477/) (2022)
- [Semantic-aware Node Synthesis for Imbalanced Heterogeneous Information Networks](https://mlsanity.com/paper/2302.14061/) (2023)
- [Decentralized Federated Learning Preserves Model and Data Privacy](https://mlsanity.com/paper/2102.00880/) (2021)
- [Impact of network topology on the performance of Decentralized Federated Learning](https://mlsanity.com/paper/2402.18606/) (2024)
- [Aergia: Leveraging Heterogeneity in Federated Learning Systems](https://mlsanity.com/paper/2210.06154/) (2022)
- [Provable Privacy Advantages of Decentralized Federated Learning via Distributed Optimization](https://mlsanity.com/paper/2407.09324/) (2024)
- [MUDGUARD: Taming Malicious Majorities in Federated Learning using Privacy-Preserving Byzantine-Robust Clustering](https://mlsanity.com/paper/2208.10161/) (2022)
- [Neighborhood Gradient Clustering: An Efficient Decentralized Learning Method for Non-IID Data Distributions](https://mlsanity.com/paper/2209.14390/) (2022)
- [Asynchronous Multi-Server Federated Learning for Geo-Distributed Clients](https://mlsanity.com/paper/2406.01439/) (2024)
- [pFedGame -- Decentralized Federated Learning using Game Theory in Dynamic Topology](https://mlsanity.com/paper/2410.04058/) (2024)

## Built on by (most cited first, 2 of 2)

- [Population Scaling or Data Dilution? Dynamics of Local Topology Evolution in Decentralized Learning](https://mlsanity.com/paper/2610.05476/) (2026)
- [LFHE: Local-First Heuristic Evolution for Bounded Local Topology Search in Decentralized Learning with Non-IID Data](https://mlsanity.com/paper/2610.08176/) (2026)

## Abstract (arXiv)

Decentralized learning (DL) enables a set of nodes to train a model collaboratively without central coordination, offering benefits for privacy and scalability. However, DL struggles to train a high accuracy model when the data distribution is non-independent and identically distributed (non-IID) and when the communication topology is static. To address these issues, we propose Morph, a topology optimization algorithm for DL. In Morph, nodes adaptively choose peers for model exchange based on maximum model dissimilarity. Morph maintains a fixed in-degree while dynamically reshaping the communication graph through gossip-based peer discovery and diversity-driven neighbor selection, thereby improving robustness to data heterogeneity. Experiments on CIFAR-10 and FEMNIST with up to 100 nodes show that Morph consistently outperforms static and epidemic baselines, while closely tracking the fully connected upper bound. On CIFAR-10, Morph achieves a relative improvement of 1.12x in test accuracy compared to the state-of-the-art baselines. On FEMNIST, Morph achieves an accuracy that is 1.08x higher than Epidemic Learning. Similar trends hold for 50 node deployments, where Morph narrows the gap to the fully connected upper bound within 0.5 percentage points on CIFAR-10. These results demonstrate that Morph achieves higher final accuracy, faster convergence, and more stable learning as quantified by lower inter-node variance, while requiring fewer communication rounds than baselines and no global knowledge.