mlsanityData to 7 Oct 2026

Improved Convergence Analysis and SNR Control Strategies for Federated Learning in the Presence of Noise

arXiv 2307.07406 ↗First posted 14 Jul 2023cs.LG cs.IT eess.SPcited by 4

Not on the map. Why?

Abstract (arXiv)

We propose an improved convergence analysis technique that characterizes the distributed learning paradigm of federated learning (FL) with imperfect/noisy uplink and downlink communications. Such imperfect communication scenarios arise in the practical deployment of FL in emerging communication systems and protocols. The analysis developed in this paper demonstrates, for the first time, that there is an asymmetry in the detrimental effects of uplink and downlink communications in FL. In particular, the adverse effect of the downlink noise is more severe on the convergence of FL algorithms. Using this insight, we propose improved Signal-to-Noise (SNR) control strategies that, discarding the negligible higher-order terms, lead to a similar convergence rate for FL as in the case of a perfect, noise-free communication channel while incurring significantly less power resources compared to existing solutions. In particular, we establish that to maintain the O(frac1√K) rate of convergence like in the case of noise-free FL, we need to scale down the uplink and downlink noise by Omega(√k) and Omega(k) respectively, where k denotes the communication round, k=1,dots, K. Our theoretical result is further characterized by two major benefits: firstly, it does not assume the somewhat unrealistic assumption of bounded client dissimilarity, and secondly, it only requires smooth non-convex loss functions, a function class better suited for modern machine learning and deep learning models. We also perform extensive empirical analysis to verify the validity of our theoretical findings.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 14 Jul 2023.

Builds on

32 of 43 references.

References · 32

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2020Adaptive Federated Optimizationcited by 1,036
  7. 2018LEAF: A Benchmark for Federated Settingscited by 780
  8. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  9. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  10. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  11. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  12. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 253
20 more
  1. 2018Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)cited by 186
  2. 2019Federated Learning over Wireless Fading Channelscited by 178
  3. 2019DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compressioncited by 98
  4. 2018SGD and Hogwild! Convergence Without the Bounded Gradients Assumptioncited by 94
  5. 2018Communication Compression for Decentralized Trainingcited by 87
  6. 2013Distributed optimization over time-varying directed graphscited by 77
  7. 2020Over-the-Air Federated Learning from Heterogeneous Datacited by 72
  8. 2020Design and Analysis of Uplink and Downlink Communications for Federated Learningcited by 48
  9. 2020Faster Non-Convex Federated Learning via Global and Local Momentumcited by 48
  10. 2019Robust Federated Learning with Noisy Communicationcited by 33
  11. 2019High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learningcited by 32
  12. 2014Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphscited by 31
  13. 2021ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Trainingcited by 22
  14. 2018Double Quantization for Communication-Efficient Distributed Optimizationcited by 18
  15. 2020Fast Convergence Algorithm for Analog Federated Learningcited by 16
  16. 2021Communication-Efficient Variance-Reduced Decentralized Stochastic Optimization over Time-Varying Directed Graphscited by 9
  17. 2021Federated Learning over Noisy Channels: Convergence Analysis and Design Examplescited by 8
  18. 2020Decentralized Optimization On Time-Varying Directed Graphs Under Communication Constraintscited by 5
  19. 2021On the Convergence of Differentially Private Federated Learning on Non-Lipschitz Objectives, and with Normalized Client Updatescited by 5
  20. 2021Physical-Layer Security via Distributed Beamforming in the Presence of Adversaries with Unknown Locationscited by 1

Built on by

Cited by year

Citing papers per year: 2024: 4.

4 papers.

  1. 2024FedNMUT -- Federated Noisy Model Update Tracking Convergence Analysiscited by 1
  2. 2024Submodular Maximization Approaches for Equitable Client Selection in Federated Learningcited by 1
  3. 2024A Hierarchical Federated Learning Approach for the Internet of Thingscited by 0
  4. 2024Equitable Federated Learning with Activation Clusteringcited by 0