mlsanityData to 7 Oct 2026

Efficient and Reliable Overlay Networks for Decentralized Federated Learning

arXiv 2112.15486 ↗First posted 12 Dec 2021cs.NI cs.DC cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

We propose near-optimal overlay networks based on d-regular expander graphs to accelerate decentralized federated learning (DFL) and improve its generalization. In DFL a massive number of clients are connected by an overlay network, and they solve machine learning problems collaboratively without sharing raw data. Our overlay network design integrates spectral graph theory and the theoretical convergence and generalization bounds for DFL. As such, our proposed overlay networks accelerate convergence, improve generalization, and enhance robustness to clients failures in DFL with theoretical guarantees. Also, we present an efficient algorithm to convert a given graph to a practical overlay network and maintaining the network topology after potential client failures. We numerically verify the advantages of DFL with our proposed networks on various benchmark tasks, ranging from image classification to language modeling using hundreds of clients.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 30 of its 59 references that are in the map. First posted on arXiv 12 Dec 2021.

Builds on

30 of 59 references.

References · 30

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 547
  7. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  8. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  9. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  10. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 208
  11. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  12. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
18 more
  1. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  2. 2020LDP-Fed: Federated Learning with Local Differential Privacycited by 129
  3. 2021Decentralized Federated Averagingcited by 114
  4. 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
  5. 2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
  6. 2014Deep learning with Elastic Averaging SGDcited by 90
  7. 2019MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Samplingcited by 67
  8. 2020Privacy For Free: Wireless Federated Learning Via Uncoded Transmission With Adaptive Power Controlcited by 62
  9. 2019Federated Learning with Bayesian Differential Privacycited by 60
  10. 2019HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learningcited by 51
  11. 2020Throughput-Optimal Topology Design for Cross-Silo Federated Learningcited by 37
  12. 2020FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selectioncited by 35
  13. 2018Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learningcited by 15
  14. 2020Anonymizing Data for Privacy-Preserving Federated Learningcited by 8
  15. 2016Expander Graph and Communication-Efficient Decentralized Optimizationcited by 7
  16. 2020A Distributed Trust Framework for Privacy-Preserving Machine Learningcited by 7
  17. 2014Fast Distributed Coordinate Descent for Non-Strongly Convex Lossescited by 4
  18. 2019Enhancing the Privacy of Federated Learning with Sketchingcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 3; 2025: 2.

9 papers.

  1. 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 137
  2. 2023Towards Understanding Generalization and Stability Gaps between Centralized and Decentralized Federated Learningcited by 3
  3. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  4. 2023Decentralized Multi-Level Compositional Optimization Algorithms with Level-Independent Convergence Ratecited by 2
  5. 2024Exposing the Vulnerability of Decentralized Learning to Membership Inference Attacks Through the Lens of Graph Mixingcited by 1
  6. 2023Secure Decentralized Learning with Blockchaincited by 0
  7. 2024Heterogeneity-Aware Resource Allocation and Topology Design for Hierarchical Federated Edge Learningcited by 0
  8. 2025Learning Decentralized Swarms Using Rotation Equivariant Graph Neural Networkscited by 0
  9. 2025Communication Optimization for Decentralized Learning atop Bandwidth-limited Edge Networkscited by 0