# Mixed-Precision Embedding Using a Cache

arXiv: https://arxiv.org/abs/2010.11305 · HTML: https://mlsanity.com/paper/2010.11305/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2020.

## Built on by (most cited first, 18 of 18)

- [Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models](https://mlsanity.com/paper/2104.05158/) (2021)
- [A Comprehensive Survey on Trustworthy Recommender Systems](https://mlsanity.com/paper/2209.10117/) (2022)
- [Accelerating Recommendation System Training by Leveraging Popular Choices](https://mlsanity.com/paper/2103.00686/) (2021)
- [Heterogeneous Acceleration Pipeline for Recommendation System Training](https://mlsanity.com/paper/2204.05436/) (2022)
- [Training Personalized Recommendation Systems from (GPU) Scratch: Look Forward not Backwards](https://mlsanity.com/paper/2205.04702/) (2022)
- [Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction](https://mlsanity.com/paper/2212.05735/) (2022)
- [Learning to Collide: Recommendation System Model Compression with Learned Hash Functions](https://mlsanity.com/paper/2203.15837/) (2022)
- [Accelerating Communication in Deep Learning Recommendation Model Training with Dual-Level Adaptive Lossy Compression](https://mlsanity.com/paper/2407.04272/) (2024)
- [CoVE: Compressed Vocabulary Expansion Makes Better LLM-based Recommender Systems](https://mlsanity.com/paper/2506.19993/) (2025)
- [On-device Content-based Recommendation with Single-shot Embedding Pruning: A Cooperative Game Perspective](https://mlsanity.com/paper/2411.13052/) (2024)
- [Fine-Grained Embedding Dimension Optimization During Training for Recommender Systems](https://mlsanity.com/paper/2401.04408/) (2024)
- [Rec-AD: An Efficient Computation Framework for FDIA Detection Based on Tensor Train Decomposition and Deep Learning Recommendation Model](https://mlsanity.com/paper/2507.14668/) (2025)
- [ECRM: Efficient Fault Tolerance for Recommendation Model Training via Erasure Coding](https://mlsanity.com/paper/2104.01981/) (2021)
- [MTrainS: Improving DLRM training efficiency using heterogeneous memories](https://mlsanity.com/paper/2305.01515/) (2023)
- [Enhancing Cross-Category Learning in Recommendation Systems with Multi-Layer Embedding Training](https://mlsanity.com/paper/2309.15881/) (2023)
- [ShuffleGate: Scalable Feature Optimization for Recommender Systems via Batch-wise Sensitivity Learning](https://mlsanity.com/paper/2503.09315/) (2025)
- [DimGrow: Memory-Efficient Field-level Embedding Dimension Search](https://mlsanity.com/paper/2505.12683/) (2025)
- [FreeScale: Distributed Training for Sequence Recommendation Models with Minimal Scaling Cost](https://mlsanity.com/paper/2604.24073/) (2026)

## Abstract (arXiv)

In recommendation systems, practitioners observed that increase in the number of embedding tables and their sizes often leads to significant improvement in model performances. Given this and the business importance of these models to major internet companies, embedding tables for personalization tasks have grown to terabyte scale and continue to grow at a significant rate. Meanwhile, these large-scale models are often trained with GPUs where high-performance memory is a scarce resource, thus motivating numerous work on embedding table compression during training. We propose a novel change to embedding tables using a cache memory architecture, where the majority of rows in an embedding is trained in low precision, and the most frequently or recently accessed rows cached and trained in full precision. The proposed architectural change works in conjunction with standard precision reduction and computer arithmetic techniques such as quantization and stochastic rounding. For an open source deep learning recommendation model (DLRM) running with Criteo-Kaggle dataset, we achieve 3x memory reduction with INT8 precision embedding tables and full-precision cache whose size are 5% of the embedding tables, while maintaining accuracy. For an industrial scale model and dataset, we achieve even higher >7x memory reduction with INT4 precision and cache size 1% of embedding tables, while maintaining accuracy, and 16% end-to-end training speedup by reducing GPU-to-host data transfers.