mlsanityData to 7 Oct 2026

Towards Bias Correction of FedAvg over Nonuniform and Time-Varying Communications

arXiv 2306.00280 ↗First posted 1 Jun 2023cs.LG cs.DC stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a decentralized learning framework wherein a parameter server (PS) and a collection of clients collaboratively train a model via minimizing a global objective. Communication bandwidth is a scarce resource; in each round, the PS aggregates the updates from a subset of clients only. In this paper, we focus on non-convex minimization that is vulnerable to non-uniform and time-varying communication failures between the PS and the clients. Specifically, in each round t, the link between the PS and client i is active with probability pit, which is textitunknown to both the PS and the clients. This arises when the channel conditions are heterogeneous across clients and are changing over time. We show that when the pit's are not uniform, textitFederated Average (FedAvg) -- the most widely adopted FL algorithm -- fails to minimize the global objective. Observing this, we propose textitFederated Postponed Broadcast (FedPBC) which is a simple variant of FedAvg. It differs from FedAvg in that the PS postpones broadcasting the global model till the end of each round. We show that FedPBC converges to a stationary point of the original objective. The introduced staleness is mild and there is no noticeable slowdown. Both theoretical analysis and numerical results are provided. On the technical front, postponing the global model broadcasts enables implicit gossiping among the clients with active links at round t. Despite pit's are time-varying, we are able to bound the perturbation of the global model dynamics via the techniques of controlling the gossip-type information mixing errors.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 14 of its 19 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

14 of 19 references.

References · 14

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  5. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  6. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  7. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  8. 2020Towards Flexible Device Participation in Federated Learningcited by 64
  9. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 62
  10. 2020Bidirectional compression in heterogeneous settings for distributed or federated learning with partial participation: tight convergence guaranteescited by 45
  11. 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45
  12. 2022On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyondcited by 37
2 more
  1. 2022Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimizationcited by 28
  2. 2023Federated Learning in the Presence of Adversarial Client Unavailabilitycited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 3.

6 papers.

  1. 2024Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailabilitycited by 9
  2. 2024Riemannian Federated Learning via Averaging Gradient Streamscited by 4
  3. 2024Empowering Federated Learning with Implicit Gossiping: Mitigating Connection Unreliability Amidst Unknown and Arbitrary Dynamicscited by 3
  4. 2026FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projectioncited by 1
  5. 2026Subspace Optimization for Efficient Federated Learning under Heterogeneous Datacited by 1
  6. 2026Over-the-Air Federated Learning in Heterogeneous Mobile Wireless Networkscited by 0