mlsanityData to 14 Oct 2026

FedScalar: Federated Learning with Scalar Communication for Bandwidth-Constrained Networks

arXiv 2410.02260 ↗First posted 3 Oct 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In bandwidth-constrained federated learning~(FL) settings, the repeated upload of high-dimensional model updates from agents to a central server constitutes the primary bottleneck, often rendering standard FL infeasible within practical communication budgets. We propose FedScalar, a communication-efficient FL algorithm in which each agent uploads only two scalar values per round, regardless of the model dimension~d. Each agent encodes its local update difference as an inner product with a locally generated random vector and transmits the resulting scalar together with the generating seed, enabling the server to reconstruct an unbiased gradient estimate without any high-dimensional transmission. We prove that FedScalar achieves a convergence rate of O(d/√K) to a stationary point for smooth non-convex loss functions, and show that adopting a Rademacher distribution for the random vector reduces the aggregation variance compared to the Gaussian case. Numerical simulations confirm that the dimension-free upload cost translates into significant improvements in wall-clock time and energy efficiency over FedAvg and QSGD in bandwidth-constrained settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 37 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

20 of 37 references.

References · 20

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  5. 2019Towards Federated Learning at Scale: System Designcited by 832
  6. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
  7. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  8. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  9. 2021A Field Guide to Federated Optimizationcited by 315
  10. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 179
  11. 2013Minimizing Finite Sums with the Stochastic Average Gradientcited by 164
  12. 2019Scheduling Policies for Federated Learning in Wireless Networkscited by 131
8 more
  1. 2020An Improved Analysis of Stochastic Gradient Descent with Momentumcited by 119
  2. 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 107
  3. 2018LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learningcited by 96
  4. 2019Communication-efficient distributed SGD with Sketchingcited by 81
  5. 2014Communication-Efficient Distributed Dual Coordinate Ascentcited by 31
  6. 2023Model-free Learning with Heterogeneous Dynamical Systems: A Federated LQR Approachcited by 23
  7. 2022Federated Learning Using Variance Reduced Stochastic Gradient for Probabilistically Activated Agentscited by 8
  8. 2024Projected Forward Gradient-Guided Frank-Wolfe Algorithm via Variance Reductioncited by 5

Built on by

Not cited yet.