# Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updates

arXiv: https://arxiv.org/abs/2105.04153 · HTML: https://mlsanity.com/paper/2105.04153/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 50 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 30 of 30)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [signSGD: Compressed Optimisation for Non-Convex Problems](https://mlsanity.com/paper/1802.04434/) (2018)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training](https://mlsanity.com/paper/1712.01887/) (2017)
- [Robust and Communication-Efficient Federated Learning from Non-IID Data](https://mlsanity.com/paper/1903.02891/) (2019)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [Federated Learning in Mobile Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/1909.11875/) (2019)
- [Sparsified SGD with Memory](https://mlsanity.com/paper/1809.07599/) (2018)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning](https://mlsanity.com/paper/1807.06629/) (2018)
- [TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning](https://mlsanity.com/paper/1705.07878/) (2017)
- [Sparse Communication for Distributed Gradient Descent](https://mlsanity.com/paper/1704.05021/) (2017)
- [The Convergence of Sparsified Gradient Methods](https://mlsanity.com/paper/1809.10505/) (2018)
- [Client-Edge-Cloud Hierarchical Federated Learning](https://mlsanity.com/paper/1905.06641/) (2019)
- [Gradient Sparsification for Communication-Efficient Distributed Optimization](https://mlsanity.com/paper/1710.09854/) (2017)
- [Distributed Mean Estimation with Limited Communication](https://mlsanity.com/paper/1611.00429/) (2016)
- [ATOMO: Communication-efficient Learning via Atomic Sparsification](https://mlsanity.com/paper/1806.04090/) (2018)
- [Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation](https://mlsanity.com/paper/1910.13067/) (2019)
- [Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach](https://mlsanity.com/paper/2001.04756/) (2020)
- [Randomized Distributed Mean Estimation: Accuracy vs Communication](https://mlsanity.com/paper/1611.07555/) (2016)
- [Federated Learning for Ultra-Reliable Low-Latency V2V Communications](https://mlsanity.com/paper/1805.09253/) (2018)
- [Millimeter Wave Channel Modeling and Cellular Capacity Evaluation](https://mlsanity.com/paper/1312.4921/) (2013)
- [Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees](https://mlsanity.com/paper/1911.08727/) (2019)
- [Federated AI for building AI Solutions across Multiple Agencies](https://mlsanity.com/paper/1809.10036/) (2018)

## Built on by (most cited first, 10 of 10)

- [Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matching](https://mlsanity.com/paper/2203.12285/) (2022)
- [Optimal Rate Adaption in Federated Learning with Compressed Communications](https://mlsanity.com/paper/2112.06694/) (2021)
- [A Fast Blockchain-based Federated Learning Framework with Compressed Communications](https://mlsanity.com/paper/2208.06095/) (2022)
- [Joint Optimization of Energy Consumption and Completion Time in Federated Learning](https://mlsanity.com/paper/2209.14900/) (2022)
- [Data-Aware Gradient Compression for FL in Communication-Constrained Mobile Computing](https://mlsanity.com/paper/2311.07324/) (2023)
- [Communication-Efficient Federated Learning through Adaptive Weight Clustering and Server-Side Distillation](https://mlsanity.com/paper/2401.14211/) (2024)
- [FedCode: Communication-Efficient Federated Learning via Transferring Codebooks](https://mlsanity.com/paper/2311.09270/) (2023)
- [γ-FedHT: Stepsize-Aware Hard-Threshold Gradient Compression in Federated Learning](https://mlsanity.com/paper/2505.12479/) (2025)
- [Taming Latency and Bandwidth: A Theoretical Framework and Adaptive Algorithm for Communication-Constrained Training](https://mlsanity.com/paper/2507.17346/) (2025)
- [Mixed-Timescale Differential Coding for Downlink Model Broadcast in Wireless Federated Learning](https://mlsanity.com/paper/2607.13119/) (2026)

## Abstract (arXiv)

Federated Learning (FL) is an emerging decentralized learning framework through which multiple clients can collaboratively train a learning model. However, a major obstacle that impedes the wide deployment of FL lies in massive communication traffic. To train high dimensional machine learning models (such as CNN models), heavy communication traffic can be incurred by exchanging model updates via the Internet between clients and the parameter server (PS), implying that the network resource can be easily exhausted. Compressing model updates is an effective way to reduce the traffic amount. However, a flexible unbiased compression algorithm applicable for both uplink and downlink compression in FL is still absent from existing works. In this work, we devise the Model Update Compression by Soft Clustering (MUCSC) algorithm to compress model updates transmitted between clients and the PS. In MUCSC, it is only necessary to transmit cluster centroids and the cluster ID of each model update. Moreover, we prove that: 1) The compressed model updates are unbiased estimation of their original values so that the convergence rate by transmitting compressed model updates is unchanged; 2) MUCSC can guarantee that the influence of the compression error on the model accuracy is minimized. Then, we further propose the boosted MUCSC (B-MUCSC) algorithm, a biased compression algorithm that can achieve an extremely high compression rate by grouping insignificant model updates into a super cluster. B-MUCSC is suitable for scenarios with very scarce network resource. Ultimately, we conduct extensive experiments with the CIFAR-10 and FEMNIST datasets to demonstrate that our algorithms can not only substantially reduce the volume of communication traffic in FL, but also improve the training efficiency in practical networks.