mlsanityData to 14 Oct 2026

Efficient Wireless Federated Learning via Low-Rank Gradient Factorization

arXiv 2401.07496 ↗First posted 15 Jan 2024cs.IT cs.LG eess.SPcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper presents a novel gradient compression method for federated learning (FL) in wireless systems. The proposed method centers on a low-rank matrix factorization strategy for local gradient compression that is based on one iteration of a distributed Jacobi successive convex approximation (SCA) at each FL round. The low-rank approximation obtained at one round is used as a "warm start" initialization for Jacobi SCA in the next FL round. A new protocol termed over-the-air low-rank compression (Ota-LC) incorporating this gradient compression method with over-the-air computation and error feedback is shown to have lower computation cost and lower communication overhead, while guaranteeing the same inference performance, as compared with existing benchmarks. As an example, when targeting a test accuracy of 70% on the Cifar-10 dataset, Ota-LC reduces total communication costs by at least 33% compared to benchmark schemes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 15 Jan 2024.

Builds on

16 of 25 references.

References · 16

  1. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  2. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  3. 2018Sparsified SGD with Memorycited by 376
  4. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 310
  5. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 238
  6. 2018The Convergence of Sparsified Gradient Methodscited by 229
  7. 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 107
  8. 2014Matrix Completion and Low-Rank SVD via Fast Alternating Least Squarescited by 104
  9. 2021Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysiscited by 51
  10. 2021Rethinking gradient sparsification as total error minimizationcited by 41
  11. 2019High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learningcited by 32
  12. 2022Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank?cited by 16
4 more
  1. 2013Decomposition by Partial Linearization: Parallel Optimization of Multi-Agent Systemscited by 7
  2. 2023LASER: Linear Compression in Wireless Distributed Optimizationcited by 6
  3. 2022Optimal MIMO Combining for Blind Federated Edge Learning with Gradient Sparsificationcited by 4
  4. 2023Over-the-Air Federated Learning Over MIMO Channels: A Sparse-Coded Multiplexing Approachcited by 2

Built on by

Not cited yet.