Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach
Not on the map. Why?
Abstract (arXiv)
Federated learning (FL) is a machine learning paradigm that targets model training without gathering the local data dispersed over various data sources. Standard FL, which employs a single server, can only support a limited number of users, leading to degraded learning capability. In this work, we consider a multi-server FL framework, referred to as Confederated Learning (CFL), in order to accommodate a larger number of users. A CFL system is composed of multiple networked edge servers, with each server connected to an individual set of users. Decentralized collaboration among servers is leveraged to harness all users' data for model training. Due to the potentially massive number of users involved, it is crucial to reduce the communication overhead of the CFL system. We propose a stochastic gradient method for distributed learning in the CFL framework. The proposed method incorporates a conditionally-triggered user selection (CTUS) mechanism as the central component to effectively reduce communication overhead. Relying on a delicately designed triggering condition, the CTUS mechanism allows each server to select only a small number of users to upload their gradients, without significantly jeopardizing the convergence performance of the algorithm. Our theoretical analysis reveals that the proposed algorithm enjoys a linear convergence rate. Simulation results show that it achieves substantial improvement over state-of-the-art algorithms in terms of communication efficiency.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 42 of its 58 references that are in the map. First posted on arXiv 28 Feb 2024.
Builds on
42 of 58 references.
References · 42
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
- 2018Local SGD Converges Fast and Communicates Littlecited by 595
- 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
- 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
- 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 306
- 2017Sparse Communication for Distributed Gradient Descentcited by 243
- 2019Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communicationcited by 218
- 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
- 2020On Biased Compression for Distributed Learningcited by 140
30 more
- 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
- 2019Scheduling Policies for Federated Learning in Wireless Networkscited by 131
- 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
- 2016NEXT: In-Network Nonconvex Optimizationcited by 107
- 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 106
- 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 103
- 2018LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learningcited by 95
- 2020Acceleration for Compressed Gradient Descent in Distributed and Federated Optimizationcited by 90
- 2020Federated Accelerated Stochastic Gradient Descentcited by 85
- 2019Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networkscited by 79
- 2018Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimizationcited by 76
- 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
- 2020Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edgecited by 71
- 2019Stochastic Distributed Learning with Gradient Quantization and Variance Reductioncited by 64
- 2020Scheduling for Cellular Federated Edge Learning with Importance and Channel Awarenesscited by 62
- 2021Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysiscited by 51
- 2018Push-Pull Gradient Methods for Distributed Optimization in Networkscited by 45
- 2020A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!cited by 41
- 2019Variance-Reduced Decentralized Stochastic Optimization with Accelerated Convergencecited by 33
- 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
- 20223PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregationcited by 27
- 2018A linear algorithm for optimization over directed graphs with geometric convergencecited by 25
- 2020Fast decentralized non-convex finite-sum optimization with recursive variance reductioncited by 17
- 2014Distributed convex optimization via continuous-time coordination algorithms with discrete-time communicationcited by 11
- 2022CompressedScaffnew: The First Theoretical Double Acceleration of Communication from Local Training and Compression in Distributed Optimizationcited by 11
- 2022Decentralized Event-Triggered Federated Learning with Heterogeneous Communication Thresholdscited by 9
- 2019Convergence of Distributed Stochastic Variance Reduced Methods without Sampling Extra Datacited by 7
- 2022Confederated Learning: Federated Learning with Decentralized Edge Serverscited by 7
- 2020Push-SAGA: A decentralized stochastic algorithm with variance reduction over directed graphscited by 6
- 2022Variance reduced stochastic optimization over directed graphs with row and column stochastic weightscited by 2
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 1.
2 papers.