mlsanityData to 7 Oct 2026

Decentralized Learning Made Easy with DecentralizePy

arXiv 2304.08322 ↗First posted 17 Apr 2023cs.DC cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Decentralized learning (DL) has gained prominence for its potential benefits in terms of scalability, privacy, and fault tolerance. It consists of many nodes that coordinate without a central server and exchange millions of parameters in the inherently iterative process of machine learning (ML) training. In addition, these nodes are connected in complex and potentially dynamic topologies. Assessing the intricate dynamics of such networks is clearly not an easy task. Often in literature, researchers resort to simulated environments that do not scale and fail to capture practical and crucial behaviors, including the ones associated to parallelism, data transfer, network delays, and wall-clock time. In this paper, we propose DecentralizePy, a distributed framework for decentralized ML, which allows for the emulation of large-scale learning networks in arbitrary topologies. We demonstrate the capabilities of DecentralizePy by deploying techniques such as sparsification and secure aggregation on top of several topologies, including dynamic networks with more than one thousand nodes.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 17 Apr 2023.

Builds on

23 of 41 references.

References · 23

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2019Towards Federated Learning at Scale: System Designcited by 826
  5. 2018LEAF: A Benchmark for Federated Settingscited by 780
  6. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  7. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  8. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  9. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  10. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  11. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  12. 2018The Convergence of Sparsified Gradient Methodscited by 228
11 more
  1. 2019Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communicationcited by 218
  2. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  3. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  4. 2019Decentralized Deep Learning with Arbitrary Communication Compressioncited by 109
  5. 2022NVIDIA FLARE: Federated Learning from Simulation to Real-Worldcited by 78
  6. 2014IPFS - Content Addressed, Versioned, P2P File Systemcited by 67
  7. 2022Topology-aware Generalization of Decentralized SGDcited by 37
  8. 2022Beyond spectral gap: The role of the topology in decentralized learningcited by 27
  9. 2021IPLS : A Framework for Decentralized Federated Learningcited by 26
  10. 2022TEE-based decentralized recommender systems: The raw data sharing redemptioncited by 6
  11. 2020Kollaps: Decentralized and Dynamic Topology Emulationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 9; 2025: 3; 2026: 4.

18 papers.

Sort:

By area:

  1. 2023Epidemic Learning: Boosting Decentralized Learning with Randomized Communicationcited by 21
  2. 2024Noiseless Privacy-Preserving Decentralized Learningcited by 9
  3. 2023Get More for Less in Decentralized Learning Systemscited by 6
  4. 2024Boosting Asynchronous Decentralized Learning with Model Fragmentationcited by 6
  5. 2024Low-Cost Privacy-Preserving Decentralized Learningcited by 4
  6. 2024Analyzing Inference Privacy Risks Through Gradients in Machine Learningcited by 4
  7. 2024Communication-Efficient Secure Aggregation in Decentralized Learningcited by 2
  8. 2024Energy-Aware Decentralized Learning with Intermittent Model Trainingcited by 1
  9. 2024Fair Decentralized Learningcited by 1
  10. 2026Heterogeneous Decentralized Diffusion Modelscited by 1
  11. 2026Your Neighbors Know: Leveraging Local Neighborhoods for Backdoor Detection in Decentralized Learningcited by 1
  12. 2024Harnessing Increased Client Participation with Cohort-Parallel Federated Learningcited by 0
6 more
  1. 2024On the Robustness of Distributed Machine Learning against Transfer Attackscited by 0
  2. 2025Accelerating MoE Model Inference with Expert Shardingcited by 0
  3. 2025Hubs and Spokes Learning: Efficient and Scalable Collaborative Machine Learningcited by 0
  4. 2025Efficient Pyramidal Analysis of Gigapixel Images on a Decentralized Modest Computer Clustercited by 0
  5. 2026Decentralized SGD with Controlled Disagreement Finds Flatter Minimacited by 0
  6. 2026Dynamic Topology Optimization for Non-IID Data in Decentralized Learningcited by 0