# Review of compressed embedding layers and their applications for recommender systems

arXiv: https://arxiv.org/abs/2306.13724 · HTML: https://mlsanity.com/paper/2306.13724/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 19 references that are in the map.
- First posted on arXiv 23 Jun 2023.

## Builds on (most cited first, 14 of 14)

- [Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems](https://mlsanity.com/paper/1909.02107/) (2019)
- [TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models](https://mlsanity.com/paper/2101.11714/) (2021)
- [Wide Compression: Tensor Ring Nets](https://mlsanity.com/paper/1802.09052/) (2018)
- [HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework](https://mlsanity.com/paper/2112.07221/) (2021)
- [RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation](https://mlsanity.com/paper/2201.10095/) (2022)
- [Tensorized Embedding Layers for Efficient Model Compression](https://mlsanity.com/paper/1901.10787/) (2019)
- [Post-Training 4-bit Quantization on Embedding Tables](https://mlsanity.com/paper/1911.02079/) (2019)
- [Accelerating Recommendation System Training by Leveraging Popular Choices](https://mlsanity.com/paper/2103.00686/) (2021)
- [Learning Compressed Embeddings for On-Device Inference](https://mlsanity.com/paper/2203.10135/) (2022)
- [Embedding Compression with Isotropic Iterative Quantization](https://mlsanity.com/paper/2001.05314/) (2020)
- [Building a Performance Model for Deep Learning Recommendation Model Training on GPUs](https://mlsanity.com/paper/2201.07821/) (2022)
- [Saec: Similarity-Aware Embedding Compression in Recommendation Systems](https://mlsanity.com/paper/1903.00103/) (2019)
- [Initialization Matters: Regularizing Manifold-informed Initialization for Neural Recommendation Systems](https://mlsanity.com/paper/2106.04993/) (2021)
- [The trade-offs of model size in large recommendation models : A 10000 × compressed criteo-tb DLRM model (100 GB parameters to mere 10MB)](https://mlsanity.com/paper/2207.10731/) (2022)

## Abstract (arXiv)

We review the literature on trainable, compressed embedding layers and discuss their applicability for compressing gigantic neural recommender systems. We also report the results we measured with our compressed embedding layers.