# Accelerating Communication in Deep Learning Recommendation Model Training with Dual-Level Adaptive Lossy Compression

arXiv: https://arxiv.org/abs/2407.04272 · HTML: https://mlsanity.com/paper/2407.04272/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 5 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [Billion-scale Commodity Embedding for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1803.02349/) (2018)
- [TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models](https://mlsanity.com/paper/2101.11714/) (2021)
- [With Shared Microexponents, A Little Shifting Goes a Long Way](https://mlsanity.com/paper/2302.08007/) (2023)
- [TTHRESH: Tensor Compression for Multidimensional Visual Data](https://mlsanity.com/paper/1806.05952/) (2018)
- [SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors](https://mlsanity.com/paper/2111.02925/) (2021)
- [Efficient Post-training Quantization with FP8 Formats](https://mlsanity.com/paper/2309.14592/) (2023)
- [Post-Training 4-bit Quantization on Embedding Tables](https://mlsanity.com/paper/1911.02079/) (2019)
- [cuSZ: An Efficient GPU-Based Error-Bounded Lossy Compression Framework for Scientific Data](https://mlsanity.com/paper/2007.09625/) (2020)
- [Mixed-Precision Embedding Using a Cache](https://mlsanity.com/paper/2010.11305/) (2020)
- [An In-Depth Analysis of the Slingshot Interconnect](https://mlsanity.com/paper/2008.08886/) (2020)
- [MGARD: A multigrid framework for high-performance, error-controlled data compression and refactoring](https://mlsanity.com/paper/2401.05994/) (2024)
- [Accelerating Multigrid-based Hierarchical Scientific Data Refactoring on GPUs](https://mlsanity.com/paper/2007.04457/) (2020)
- [FZ-GPU: A Fast and High-Ratio Lossy Compressor for Scientific Computing Applications on GPUs](https://mlsanity.com/paper/2304.12557/) (2023)
- [GPULZ: Optimizing LZSS Lossless Compression for Multi-byte Data on Modern GPUs](https://mlsanity.com/paper/2304.07342/) (2023)

## Built on by (most cited first, 5 of 5)

- [The Big Send-off: Scalable and Performant Collectives for Deep Learning](https://mlsanity.com/paper/2504.18658/) (2025)
- [NestPipe: Large-Scale Recommendation Training on 1,500+ Accelerators via Nested Pipelining](https://mlsanity.com/paper/2604.06956/) (2026)
- [When KV Meets Embeddings: Dynamic GPU Memory Allocation for Accelerating Generative Recommender Serving](https://mlsanity.com/paper/2605.04450/) (2026)
- [NCCLZ: Compression-Enabled GPU Collectives with Decoupled Quantization and Entropy Coding](https://mlsanity.com/paper/2605.12396/) (2026)
- [Reducing the GPU Memory Bottleneck with Lossless Compression for ML -- Extended](https://mlsanity.com/paper/2605.30728/) (2026)

## Abstract (arXiv)

DLRM is a state-of-the-art recommendation system model that has gained widespread adoption across various industry applications. The large size of DLRM models, however, necessitates the use of multiple devices/GPUs for efficient training. A significant bottleneck in this process is the time-consuming all-to-all communication required to collect embedding data from all devices. To mitigate this, we introduce a method that employs error-bounded lossy compression to reduce the communication data size and accelerate DLRM training. We develop a novel error-bounded lossy compression algorithm, informed by an in-depth analysis of embedding data features, to achieve high compression ratios. Moreover, we introduce a dual-level adaptive strategy for error-bound adjustment, spanning both table-wise and iteration-wise aspects, to balance the compression benefits with the potential impacts on accuracy. We further optimize our compressor for PyTorch tensors on GPUs, minimizing compression overhead. Evaluation shows that our method achieves a 1.38× training speedup with a minimal accuracy impact.