# Communication-Efficient Federated Learning with Dual-Side Low-Rank Compression

arXiv: https://arxiv.org/abs/2104.12416 · HTML: https://mlsanity.com/paper/2104.12416/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 16 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 12 of 12)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [Sparse Communication for Distributed Gradient Descent](https://mlsanity.com/paper/1704.05021/) (2017)
- [PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization](https://mlsanity.com/paper/1905.13727/) (2019)
- [Model Pruning Enables Efficient Federated Learning on Edge Devices](https://mlsanity.com/paper/1909.12326/) (2019)
- [Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning](https://mlsanity.com/paper/1810.01075/) (2018)
- [Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations](https://mlsanity.com/paper/1906.02367/) (2019)
- [On Generalization Bounds of a Family of Recurrent Neural Networks](https://mlsanity.com/paper/1910.12947/) (2019)
- [Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning](https://mlsanity.com/paper/2011.04050/) (2020)
- [TRP: Trained Rank Pruning for Efficient Deep Neural Networks](https://mlsanity.com/paper/2004.14566/) (2020)

## Built on by (most cited first, 10 of 10)

- [FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2108.06098/) (2021)
- [FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorization](https://mlsanity.com/paper/2111.14655/) (2021)
- [Federated Dynamical Low-Rank Training with Global Loss Convergence Guarantees](https://mlsanity.com/paper/2406.17887/) (2024)
- [Communication-Efficient Federated Low-Rank Update Algorithm and its Connection to Implicit Regularization](https://mlsanity.com/paper/2409.12371/) (2024)
- [Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspective](https://mlsanity.com/paper/2405.20431/) (2024)
- [Optimal Batch-Size Control for Low-Latency Federated Learning with Device Heterogeneity](https://mlsanity.com/paper/2507.15601/) (2025)
- [FedNet2Net: Saving Communication and Computations in Federated Learning with Model Growing](https://mlsanity.com/paper/2207.09568/) (2022)
- [On the Convergence and Stability of Distributed Sub-model Training](https://mlsanity.com/paper/2511.06132/) (2025)
- [Federated Learning for Energy Constrained IoT devices: A systematic mapping study](https://mlsanity.com/paper/2301.03720/) (2023)
- [Timely Asynchronous Hierarchical Federated Learning: Age of Convergence](https://mlsanity.com/paper/2306.12400/) (2023)

## Abstract (arXiv)

Federated learning (FL) is a promising and powerful approach for training deep learning models without sharing the raw data of clients. During the training process of FL, the central server and distributed clients need to exchange a vast amount of model information periodically. To address the challenge of communication-intensive training, we propose a new training method, referred to as federated learning with dual-side low-rank compression (FedDLR), where the deep learning model is compressed via low-rank approximations at both the server and client sides. The proposed FedDLR not only reduces the communication overhead during the training stage but also directly generates a compact model to speed up the inference process. We shall provide convergence analysis, investigate the influence of the key parameters, and empirically show that FedDLR outperforms the state-of-the-art solutions in terms of both the communication and computation efficiency.