Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learning
Not on the map. Why?
Abstract (arXiv)
Expensive communication cost is a common performance bottleneck in Federated Learning (FL), which makes it less appealing in real-world applications. Many communication-efficient FL methods focus on discarding a part of model updates mostly based on gradient magnitude. In this study, we find that recycling previous updates, rather than simply dropping them, more effectively reduces the communication cost while maintaining FL performance. We propose FedLUAR, a Layer-wise Update Aggregation with Recycling scheme for communication-efficient FL. We first define a useful metric that quantifies the extent to which the aggregated gradients influences the model parameter values in each layer. FedLUAR selects a few layers based on the metric and recycles their previous updates on the server side. Our extensive empirical study demonstrates that the update recycling scheme significantly reduces the communication cost while maintaining model accuracy. For example, our method achieves nearly the same AG News accuracy as FedAvg, while reducing the communication cost to just 17%.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 45 references that are in the map. First posted on arXiv 14 Mar 2025.
Builds on
35 of 45 references.
References · 35
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
- 2016Wide Residual Networkscited by 3,070
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2020Adaptive Federated Optimizationcited by 1,046
- 2018LEAF: A Benchmark for Federated Settingscited by 792
- 2021Model-Contrastive Federated Learningcited by 766
- 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 674
- 2017Large Batch Training of Convolutional Networkscited by 535
23 more
- 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
- 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 256
- 2018The Convergence of Sparsified Gradient Methodscited by 229
- 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
- 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
- 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 187
- 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 160
- 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 141
- 2020Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Networkcited by 74
- 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 55
- 2021Layer-wise Adaptive Model Aggregation for Scalable Federated Learningcited by 36
- 2020PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learningcited by 35
- 2018Efficient Decentralized Deep Learning by Dynamic Model Averagingcited by 30
- 2020Extrapolation for Large-batch Training in Deep Learningcited by 25
- 2022Communication-Efficient Federated Learning with Accelerated Client Gradientcited by 24
- 2022Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank?cited by 16
- 2023Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devicescited by 11
- 2024SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overheadcited by 9
- 2024FedBAT: Communication-Efficient Federated Learning via Learnable Binarizationcited by 9
- 2021FedDropoutAvg: Generalizable federated learning for histopathology image classificationcited by 7
- 2023Model Pruning Enables Localized and Efficient Federated Learning for Yield Forecasting and Data Sharingcited by 7
- 2023On the Weight Dynamics of Deep Normalized Networkscited by 4
- 2024Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learningcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.