Reducing Training Time in Cross-Silo Federated Learning using Multigraph Topology
Not on the map. Why?
Abstract (arXiv)
Federated learning is an active research topic since it enables several participants to jointly train a model without sharing local data. Currently, cross-silo federated learning is a popular training setting that utilizes a few hundred reliable data silos with high-speed access links to training a model. While this approach has been widely applied in real-world scenarios, designing a robust topology to reduce the training time remains an open problem. In this paper, we present a new multigraph topology for cross-silo federated learning. We first construct the multigraph using the overlay graph. We then parse this multigraph into different simple graphs with isolated nodes. The existence of isolated nodes allows us to perform model aggregation without waiting for other nodes, hence effectively reducing the training time. Intensive experiments on three public datasets show that our proposed method significantly reduces the training time compared with recent state-of-the-art topologies while maintaining the accuracy of the learned model. Our code can be found at https://github.com/aioz-ai/MultigraphFL
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 51 of its 99 references that are in the map. First posted on arXiv 20 Jul 2022.
Builds on
51 of 99 references.
References · 51
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
- 2018LEAF: A Benchmark for Federated Settingscited by 780
- 2021Model-Contrastive Federated Learningcited by 762
- 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
- 2020An Efficient Framework for Clustered Federated Learningcited by 505
- 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
- 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
38 more
- 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 284
- 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
- 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
- 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
- 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2017Asynchronous Decentralized Parallel Stochastic Gradient Descentcited by 159
- 2021Subgraph Federated Learning with Missing Neighbor Generationcited by 132
- 2017Network Topology and Communication-Computation Tradeoffs in Decentralized Optimizationcited by 119
- 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
- 2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
- 2022Layer-wised Model Aggregation for Personalized Federated Learningcited by 95
- 2013Distributed optimization over time-varying directed graphscited by 77
- 2016CoCoA: A General Framework for Communication-Efficient Distributed Optimizationcited by 69
- 2019MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Samplingcited by 67
- 2017Collaborative Deep Learning in Fixed Topology Networkscited by 60
- 2021Deep Federated Learning for Autonomous Drivingcited by 54
- 2020Wireless Communications for Collaborative Federated Learningcited by 47
- 2019Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approachcited by 45
- 2020Decentralized Federated Learning via Mutual Knowledge Transfercited by 42
- 2021RelaySum for Decentralized Deep Learning on Heterogeneous Datacited by 39
- 2020Throughput-Optimal Topology Design for Cross-Silo Federated Learningcited by 37
- 2022Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customizationcited by 34
- 2019Central Server Free Federated Learning over Single-sided Trust Social Networkscited by 33
- 2014Communication-Efficient Distributed Dual Coordinate Ascentcited by 30
- 2018COLA: Decentralized Linear Learningcited by 30
- 2018Pipe-SGD: A Decentralized Pipelined SGD Framework for Distributed Deep Net Trainingcited by 28
- 2019Multi-graph Fusion for Multi-view Spectral Clusteringcited by 27
- 2017Straggler Mitigation in Distributed Optimization Through Data Encodingcited by 26
- 2021Distributed Dynamic Map Fusion via Federated Learning for Intelligent Networked Vehiclescited by 24
- 2022FedNew: A Communication-Efficient and Privacy-Preserving Newton-Type Method for Federated Learningcited by 23
- 2015Adding vs. Averaging in Distributed Primal-Dual Optimizationcited by 19
- 2018Multi-View Multi-Graph Embedding for Brain Network Clustering Analysiscited by 14
- 2018Towards More Efficient Stochastic Decentralized Learning: Faster Convergence and Sparse Communicationcited by 13
- 2017Near-Optimal Straggler Mitigation for Distributed Gradient Methodscited by 11
- 2022Tackling Data Heterogeneity: A New Unified Framework for Decentralized SGD with Sample-induced Topologycited by 10
- 2021Brain Multigraph Prediction using Topology-Aware Adversarial Graph Neural Networkcited by 7
- 2019Learning Cross-Domain Representation with Multi-Graph Neural Networkcited by 5
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.