# FEDZIP: A Compression Framework for Communication-Efficient Federated Learning

arXiv: https://arxiv.org/abs/2102.01593 · HTML: https://mlsanity.com/paper/2102.01593/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 55 references that are in the map.
- First posted on arXiv 2 Feb 2021.

## Builds on (most cited first, 29 of 29)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [signSGD: Compressed Optimisation for Non-Convex Problems](https://mlsanity.com/paper/1802.04434/) (2018)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training](https://mlsanity.com/paper/1712.01887/) (2017)
- [Differentially Private Federated Learning: A Client Level Perspective](https://mlsanity.com/paper/1712.07557/) (2017)
- [Robust and Communication-Efficient Federated Learning from Non-IID Data](https://mlsanity.com/paper/1903.02891/) (2019)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning](https://mlsanity.com/paper/1705.07878/) (2017)
- [Sparse Communication for Distributed Gradient Descent](https://mlsanity.com/paper/1704.05021/) (2017)
- [Federated Optimization:Distributed Optimization Beyond the Datacenter](https://mlsanity.com/paper/1511.03575/) (2015)
- [Revisiting Distributed Synchronous SGD](https://mlsanity.com/paper/1702.05800/) (2017)
- [ATOMO: Communication-efficient Learning via Atomic Sparsification](https://mlsanity.com/paper/1806.04090/) (2018)
- [Federated Learning for Keyword Spotting](https://mlsanity.com/paper/1810.05512/) (2018)
- [DeepCache: Principled Cache for Mobile Deep Vision](https://mlsanity.com/paper/1712.01670/) (2017)
- [Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication](https://mlsanity.com/paper/1805.08768/) (2018)
- [A First Look at Deep Learning Apps on Smartphones](https://mlsanity.com/paper/1812.05448/) (2018)
- [A Distributed Synchronous SGD Algorithm with Global Top-k Sparsification for Low Bandwidth Networks](https://mlsanity.com/paper/1901.04359/) (2019)
- [Efficient Decentralized Deep Learning by Dynamic Model Averaging](https://mlsanity.com/paper/1807.03210/) (2018)
- [Mask-RCNN and U-net Ensembled for Nuclei Segmentation](https://mlsanity.com/paper/1901.10170/) (2019)
- [Variance-based Gradient Compression for Efficient Distributed Deep Learning](https://mlsanity.com/paper/1802.06058/) (2018)
- [CovidCTNet: An Open-Source Deep Learning Approach to Identify Covid-19 Using CT Image](https://mlsanity.com/paper/2005.03059/) (2020)
- [SEFR: A Fast Linear-Time Classifier for Ultra-Low Power Devices](https://mlsanity.com/paper/2006.04620/) (2020)
- [Nested Dithered Quantization for Communication Reduction in Distributed Training](https://mlsanity.com/paper/1904.01197/) (2019)

## Built on by (most cited first, 27 of 27)

- [Stochastic Controlled Averaging for Federated Learning with Communication Compression](https://mlsanity.com/paper/2308.08165/) (2023)
- [From Distributed Machine Learning to Federated Learning: A Survey](https://mlsanity.com/paper/2104.14362/) (2021)
- [DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning](https://mlsanity.com/paper/2111.00465/) (2021)
- [AdaSplit: Adaptive Trade-offs for Resource-constrained Distributed Deep Learning](https://mlsanity.com/paper/2112.01637/) (2021)
- [Sparsified Secure Aggregation for Privacy-Preserving Federated Learning](https://mlsanity.com/paper/2112.12872/) (2021)
- [Federated Progressive Sparsification (Purge, Merge, Tune)+](https://mlsanity.com/paper/2204.12430/) (2022)
- [Genetic CFL: Optimization of Hyper-Parameters in Clustered Federated Learning](https://mlsanity.com/paper/2107.07233/) (2021)
- [Data-Aware Gradient Compression for FL in Communication-Constrained Mobile Computing](https://mlsanity.com/paper/2311.07324/) (2023)
- [Cerberus: Exploring Federated Prediction of Security Events](https://mlsanity.com/paper/2209.03050/) (2022)
- [Communication-Efficient Federated Learning through Adaptive Weight Clustering and Server-Side Distillation](https://mlsanity.com/paper/2401.14211/) (2024)
- [SplitFedZip: Learned Compression for Data Transfer Reduction in Split-Federated Learning](https://mlsanity.com/paper/2412.17150/) (2024)
- [Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression](https://mlsanity.com/paper/2211.14292/) (2022)
- [Adaptive Parameterization of Deep Learning Models for Federated Learning](https://mlsanity.com/paper/2302.02949/) (2023)
- [FedCode: Communication-Efficient Federated Learning via Transferring Codebooks](https://mlsanity.com/paper/2311.09270/) (2023)
- [FedRN: Exploiting k-Reliable Neighbors Towards Robust Federated Learning](https://mlsanity.com/paper/2205.01310/) (2022)
- [Communication Optimization in Large Scale Federated Learning using Autoencoder Compressed Weight Updates](https://mlsanity.com/paper/2108.05670/) (2021)
- [γ-FedHT: Stepsize-Aware Hard-Threshold Gradient Compression in Federated Learning](https://mlsanity.com/paper/2505.12479/) (2025)
- [EFU: Enforcing Federated Unlearning via Functional Encryption](https://mlsanity.com/paper/2508.07873/) (2025)
- [Federated Learning for Energy Constrained IoT devices: A systematic mapping study](https://mlsanity.com/paper/2301.03720/) (2023)
- [FedShift: Robust Federated Learning Aggregation Scheme in Resource Constrained Environment via Weight Shifting](https://mlsanity.com/paper/2402.01070/) (2024)
- [Federated Computing -- Survey on Building Blocks, Extensions and Systems](https://mlsanity.com/paper/2404.02779/) (2024)
- [A Robust Federated Learning Framework for Undependable Devices at Scale](https://mlsanity.com/paper/2412.19991/) (2024)
- [Federated Low-Rank Tensor Estimation for Multimodal Image Reconstruction](https://mlsanity.com/paper/2502.02761/) (2025)
- [Leveraging Label Potential for Enhanced Multimodal Emotion Recognition](https://mlsanity.com/paper/2504.05158/) (2025)
- [RefProtoFL: Communication-Efficient Federated Learning via External-Referenced Prototype Alignment](https://mlsanity.com/paper/2601.14746/) (2026)
- [A Full Compression Pipeline for Green Federated Learning in Communication-Constrained Environments](https://mlsanity.com/paper/2604.11146/) (2026)
- [HybridFLow: SDN-Orchestrated Client Partitioning for Hybrid Federated Learning](https://mlsanity.com/paper/2609.10404/) (2026)

## Abstract (arXiv)

Federated Learning marks a turning point in the implementation of decentralized machine learning (especially deep learning) for wireless devices by protecting users' privacy and safeguarding raw data from third-party access. It assigns the learning process independently to each client. First, clients locally train a machine learning model based on local data. Next, clients transfer local updates of model weights and biases (training data) to a server. Then, the server aggregates updates (received from clients) to create a global learning model. However, the continuous transfer between clients and the server increases communication costs and is inefficient from a resource utilization perspective due to the large number of parameters (weights and biases) used by deep learning models. The cost of communication becomes a greater concern when the number of contributing clients and communication rounds increases. In this work, we propose a novel framework, FedZip, that significantly decreases the size of updates while transferring weights from the deep learning model between clients and their servers. FedZip implements Top-z sparsification, uses quantization with clustering, and implements compression with three different encoding methods. FedZip outperforms state-of-the-art compression frameworks and reaches compression rates up to 1085x, and preserves up to 99% of bandwidth and 99% of energy for clients during communication.