# Ternary Compression for Communication-Efficient Federated Learning

arXiv: https://arxiv.org/abs/2003.03564 · HTML: https://mlsanity.com/paper/2003.03564/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2020.

## Built on by (most cited first, 30 of 57)

- [Federated Learning on Non-IID Data: A Survey](https://mlsanity.com/paper/2106.06843/) (2021)
- [Distributed Learning in Wireless Networks: Recent Progress and Future Challenges](https://mlsanity.com/paper/2104.02151/) (2021)
- [From Distributed Machine Learning to Federated Learning: A Survey](https://mlsanity.com/paper/2104.14362/) (2021)
- [Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learning](https://mlsanity.com/paper/2312.17493/) (2023)
- [FedJAX: Federated learning simulation with JAX](https://mlsanity.com/paper/2108.02117/) (2021)
- [Knowledge Distillation for Federated Learning: a Practical Guide](https://mlsanity.com/paper/2211.04742/) (2022)
- [FS-Real: Towards Real-World Cross-Device Federated Learning](https://mlsanity.com/paper/2303.13363/) (2023)
- [FedDQ: Communication-Efficient Federated Learning with Descending Quantization](https://mlsanity.com/paper/2110.02291/) (2021)
- [Communication-Efficient Split Learning via Adaptive Feature-Wise Compression](https://mlsanity.com/paper/2307.10805/) (2023)
- [Adaptive Transmission Scheduling in Wireless Networks for Asynchronous Federated Learning](https://mlsanity.com/paper/2103.01422/) (2021)
- [Green, Quantized Federated Learning over Wireless Networks: An Energy-Efficient Design](https://mlsanity.com/paper/2207.09387/) (2022)
- [Communication-Efficient Federated Learning with Binary Neural Networks](https://mlsanity.com/paper/2110.02226/) (2021)
- [Hypernetwork-based Personalized Federated Learning for Multi-Institutional CT Imaging](https://mlsanity.com/paper/2206.03709/) (2022)
- [HCFL: A High Compression Approach for Communication-Efficient Federated Learning in Very Large Scale IoT Networks](https://mlsanity.com/paper/2204.06760/) (2022)
- [Green Federated Learning: A new era of Green Aware AI](https://mlsanity.com/paper/2409.12626/) (2024)
- [Temporal-Structure-Assisted Gradient Aggregation for Over-the-Air Federated Edge Learning](https://mlsanity.com/paper/2103.02270/) (2021)
- [Over-the-Air Computation Based on Balanced Number Systems for Federated Edge Learning](https://mlsanity.com/paper/2210.07012/) (2022)
- [Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data](https://mlsanity.com/paper/2203.01752/) (2022)
- [A Federated Data-Driven Evolutionary Algorithm](https://mlsanity.com/paper/2102.08288/) (2021)
- [Sparsified Secure Aggregation for Privacy-Preserving Federated Learning](https://mlsanity.com/paper/2112.12872/) (2021)
- [Federated Progressive Sparsification (Purge, Merge, Tune)+](https://mlsanity.com/paper/2204.12430/) (2022)
- [Optimising Communication Overhead in Federated Learning Using NSGA-II](https://mlsanity.com/paper/2204.02183/) (2022)
- [EvoFed: Leveraging Evolutionary Strategies for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2311.07485/) (2023)
- [Communication-Efficient Hybrid Federated Learning for E-health with Horizontal and Vertical Data Partitioning](https://mlsanity.com/paper/2404.10110/) (2024)
- [A Federated Data-Driven Evolutionary Algorithm for Expensive Multi/Many-objective Optimization](https://mlsanity.com/paper/2106.12086/) (2021)
- [FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization](https://mlsanity.com/paper/2406.18156/) (2024)
- [ResFed: Communication Efficient Federated Learning by Transmitting Deep Compressed Residuals](https://mlsanity.com/paper/2212.05602/) (2022)
- [Federated Neural Radiance Fields](https://mlsanity.com/paper/2305.01163/) (2023)
- [Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers](https://mlsanity.com/paper/2403.14716/) (2024)
- [TinyMLOps: Operational Challenges for Widespread Edge AI Adoption](https://mlsanity.com/paper/2203.10923/) (2022)

## Abstract (arXiv)

Learning over massive data stored in different locations is essential in many real-world applications. However, sharing data is full of challenges due to the increasing demands of privacy and security with the growing use of smart mobile devices and IoT devices. Federated learning provides a potential solution to privacy-preserving and secure machine learning, by means of jointly training a global model without uploading data distributed on multiple devices to a central server. However, most existing work on federated learning adopts machine learning models with full-precision weights, and almost all these models contain a large number of redundant parameters that do not need to be transmitted to the server, consuming an excessive amount of communication costs. To address this issue, we propose a federated trained ternary quantization (FTTQ) algorithm, which optimizes the quantized networks on the clients through a self-learning quantization factor. Theoretical proofs of the convergence of quantization factors, unbiasedness of FTTQ, as well as a reduced weight divergence are given. On the basis of FTTQ, we propose a ternary federated averaging protocol (T-FedAvg) to reduce the upstream and downstream communication of federated learning systems. Empirical experiments are conducted to train widely used deep learning models on publicly available datasets, and our results demonstrate that the proposed T-FedAvg is effective in reducing communication costs and can even achieve slightly better performance on non-IID data in contrast to the canonical federated learning algorithms.