# 3LC: Lightweight and Effective Traffic Compression for Distributed Machine Learning

arXiv: https://arxiv.org/abs/1802.07389 · HTML: https://mlsanity.com/paper/1802.07389/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2018.

## Built on by (most cited first, 16 of 16)

- [Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models](https://mlsanity.com/paper/2104.05158/) (2021)
- [DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training](https://mlsanity.com/paper/2206.00187/) (2022)
- [Distributed Learning Systems with First-order Methods](https://mlsanity.com/paper/2104.05245/) (2021)
- [Optimizing the Communication-Accuracy Trade-off in Federated Learning with Rate-Distortion Theory](https://mlsanity.com/paper/2201.02664/) (2022)
- [Correlated quantization for distributed mean estimation and optimization](https://mlsanity.com/paper/2203.04925/) (2022)
- [Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2404.06114/) (2024)
- [LASER: Linear Compression in Wireless Distributed Optimization](https://mlsanity.com/paper/2310.13033/) (2023)
- [CGX: Adaptive System Support for Communication-Efficient Deep Learning](https://mlsanity.com/paper/2111.08617/) (2021)
- [Correlated Quantization for Faster Nonconvex Distributed Optimization](https://mlsanity.com/paper/2401.05518/) (2024)
- [DeepReduce: A Sparse-tensor Communication Framework for Distributed Deep Learning](https://mlsanity.com/paper/2102.03112/) (2021)
- [FastSGD: A Fast Compressed SGD Framework for Distributed Machine Learning](https://mlsanity.com/paper/2112.04291/) (2021)
- [Enhancing Efficiency in Multidevice Federated Learning through Data Selection](https://mlsanity.com/paper/2211.04175/) (2022)
- [MergeComp: A Compression Scheduler for Scalable Communication-Efficient Distributed Training](https://mlsanity.com/paper/2103.15195/) (2021)
- [L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learning](https://mlsanity.com/paper/2210.17357/) (2022)
- [CrossoverScheduler: Overlapping Multiple Distributed Training Applications in a Crossover Manner](https://mlsanity.com/paper/2103.07974/) (2021)
- [Compressed Proximal Federated Learning for Non-Convex Composite Optimization on Heterogeneous Data](https://mlsanity.com/paper/2603.07654/) (2026)

## Abstract (arXiv)

The performance and efficiency of distributed machine learning (ML) depends significantly on how long it takes for nodes to exchange state changes. Overly-aggressive attempts to reduce communication often sacrifice final model accuracy and necessitate additional ML techniques to compensate for this loss, limiting their generality. Some attempts to reduce communication incur high computation overhead, which makes their performance benefits visible only over slow networks. We present 3LC, a lossy compression scheme for state change traffic that strikes balance between multiple goals: traffic reduction, accuracy, computation overhead, and generality. It combines three new techniques---3-value quantization with sparsity multiplication, quartic encoding, and zero-run encoding---to leverage strengths of quantization and sparsification techniques and avoid their drawbacks. It achieves a data compression ratio of up to 39--107X, almost the same test accuracy of trained models, and high compression speed. Distributed ML frameworks can employ 3LC without modifications to existing ML algorithms. Our experiments show that 3LC reduces wall-clock training time of ResNet-110--based image classifiers for CIFAR-10 on a 10-GPU cluster by up to 16--23X compared to TensorFlow's baseline design.