# CosSGD: Communication-Efficient Federated Learning with a Simple Cosine-Based Quantization

arXiv: https://arxiv.org/abs/2012.08241 · HTML: https://mlsanity.com/paper/2012.08241/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2020.

## Built on by (most cited first, 6 of 6)

- [ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training](https://mlsanity.com/paper/2110.05323/) (2021)
- [FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations](https://mlsanity.com/paper/2302.01068/) (2023)
- [Language Models as Zero-shot Lossless Gradient Compressors: Towards General Neural Parameter Prior Models](https://mlsanity.com/paper/2409.17836/) (2024)
- [Feature-based Federated Transfer Learning: Communication Efficiency, Robustness and Privacy](https://mlsanity.com/paper/2405.09014/) (2024)
- [Towards Federated Learning with On-device Training and Communication in 8-bit Floating Point](https://mlsanity.com/paper/2407.02610/) (2024)
- [HybridFLow: SDN-Orchestrated Client Partitioning for Hybrid Federated Learning](https://mlsanity.com/paper/2609.10404/) (2026)

## Abstract (arXiv)

Federated learning is a promising framework to mitigate data privacy and computation concerns. However, the communication cost between the server and clients has become the major bottleneck for successful deployment. Despite notable progress in gradient compression, the existing quantization methods require further improvement when low-bits compression is applied, especially the overall systems often degenerate a lot when quantization are applied in double directions to compress model weights and gradients. In this work, we propose a simple cosine-based nonlinear quantization and achieve impressive results in compressing round-trip communication costs. We are not only able to compress model weights and gradients at higher ratios than previous methods, but also achieve competing model performance at the same time. Further, our approach is highly suitable for federated learning problems since it has low computational complexity and requires only a little additional data to recover the compressed information. Extensive experiments have been conducted on image classification and brain tumor semantic segmentation using the CIFAR-10, and BraTS datasets where we show state-of-the-art effectiveness and impressive communication efficiency.