mlsanityData to 7 Oct 2026

Communication-Efficient Federated Learning with Dual-Side Low-Rank Compression

arXiv 2104.12416 ↗First posted 26 Apr 2021cs.LG cs.DCcited by 10

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a promising and powerful approach for training deep learning models without sharing the raw data of clients. During the training process of FL, the central server and distributed clients need to exchange a vast amount of model information periodically. To address the challenge of communication-intensive training, we propose a new training method, referred to as federated learning with dual-side low-rank compression (FedDLR), where the deep learning model is compressed via low-rank approximations at both the server and client sides. The proposed FedDLR not only reduces the communication overhead during the training stage but also directly generates a compact model to speed up the inference process. We shall provide convergence analysis, investigate the influence of the key parameters, and empirically show that FedDLR outperforms the state-of-the-art solutions in terms of both the communication and computation efficiency.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 12 of its 16 references that are in the map. First posted on arXiv 26 Apr 2021.

Builds on

12 of 16 references.

References · 12

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  4. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  5. 2017Sparse Communication for Distributed Gradient Descentcited by 243
  6. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 234
  7. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  8. 2018Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learningcited by 211
  9. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 179
  10. 2019On Generalization Bounds of a Family of Recurrent Neural Networkscited by 42
  11. 2020Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learningcited by 37
  12. 2020TRP: Trained Rank Pruning for Efficient Deep Neural Networkscited by 32

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 1; 2023: 2; 2024: 3; 2025: 2.

10 papers.

  1. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 140
  2. 2021FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorizationcited by 31
  3. 2024Federated Dynamical Low-Rank Training with Global Loss Convergence Guaranteescited by 7
  4. 2024Communication-Efficient Federated Low-Rank Update Algorithm and its Connection to Implicit Regularizationcited by 6
  5. 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspectivecited by 4
  6. 2025Optimal Batch-Size Control for Low-Latency Federated Learning with Device Heterogeneitycited by 2
  7. 2022FedNet2Net: Saving Communication and Computations in Federated Learning with Model Growingcited by 1
  8. 2025On the Convergence and Stability of Distributed Sub-model Trainingcited by 1
  9. 2023Federated Learning for Energy Constrained IoT devices: A systematic mapping studycited by 0
  10. 2023Timely Asynchronous Hierarchical Federated Learning: Age of Convergencecited by 0