# ProactivePIM: Accelerating Weight-Sharing Embedding Layer with PIM for Scalable Recommendation System

arXiv: https://arxiv.org/abs/2402.04032 · HTML: https://mlsanity.com/paper/2402.04032/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 66 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 20 of 20)

- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [MLPerf Inference Benchmark](https://mlsanity.com/paper/1911.02549/) (2019)
- [The Architectural Implications of Facebook's DNN-based Personalized Recommendation](https://mlsanity.com/paper/1906.03109/) (2019)
- [Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models](https://mlsanity.com/paper/2104.05158/) (2021)
- [Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems](https://mlsanity.com/paper/1909.02107/) (2019)
- [TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models](https://mlsanity.com/paper/2101.11714/) (2021)
- [RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing](https://mlsanity.com/paper/1912.12953/) (2019)
- [AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations](https://mlsanity.com/paper/2002.11252/) (2020)
- [TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learning](https://mlsanity.com/paper/1908.03072/) (2019)
- [Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems](https://mlsanity.com/paper/1909.11810/) (2019)
- [Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training](https://mlsanity.com/paper/2108.09373/) (2021)
- [RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation](https://mlsanity.com/paper/2201.10095/) (2022)
- [Post-Training 4-bit Quantization on Embedding Tables](https://mlsanity.com/paper/1911.02079/) (2019)
- [OptEmbed: Learning Optimal Embedding Table for Click-through Rate Prediction](https://mlsanity.com/paper/2208.04482/) (2022)
- [Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction](https://mlsanity.com/paper/2212.05735/) (2022)
- [Benchmarking a New Paradigm: An Experimental Analysis of a Real Processing-in-Memory Architecture](https://mlsanity.com/paper/2105.03814/) (2021)
- [Hercules: Heterogeneity-Aware Inference Serving for At-Scale Personalized Recommendation](https://mlsanity.com/paper/2203.07424/) (2022)
- [Accelerating Bandwidth-Bound Deep Learning Inference with Main-Memory Accelerators](https://mlsanity.com/paper/2012.00158/) (2020)
- [Supporting Massive DLRM Inference Through Software Defined Memory](https://mlsanity.com/paper/2110.11489/) (2021)
- [Fine-Grained Embedding Dimension Optimization During Training for Recommender Systems](https://mlsanity.com/paper/2401.04408/) (2024)

## Abstract (arXiv)

Although deep learning-based personalized recommendation systems provide qualified recommendations, they strain data center resources. The main bottleneck is the embedding layer, which is highly memory-intensive due to its sparse, irregular access patterns to embeddings. Recent near-memory processing (NMP) and processing-in-memory (PIM) architectures have addressed these issues by exploiting parallelism within memory. However, as model sizes increase year by year and can exceed server capacity, inference on single-node servers becomes challenging, necessitating the integration of model compression. Various algorithms have been proposed for model size reduction, but they come at the cost of increased memory access and CPU-PIM communication. We present ProactivePIM, a PIM system tailored for weight-sharing algorithms, a family of compression methods that decompose an embedding table into compact subtables, such as QR-trick and TT-Rec. Our analysis shows that embedding layer execution with weight-sharing algorithms increases memory access and incurs CPU-PIM communication. We also find that these algorithms exhibit unique data locality characteristics, which we name intra-GnR locality. ProactivePIM accelerates weight-sharing algorithms by utilizing a heterogeneous HBM-DIMM memory architecture with integration of a two-level PIM system of base-die PIM (bd-PIM) and bank-group PIM (bg-PIM) inside the HBM. To gain further speedup, ProactivePIM prefetches embeddings with high intra-GnR locality into an SRAM cache within bg-PIM and eliminates the CPU-PIM communication through duplication of target subtables across bank groups. With additional optimization techniques, our design effectively accelerates weight-sharing algorithms, achieving 2.22x and 2.15x speedup in QR-trick and TT-Rec, respectively, compared to the baseline architecture.