# Disaggregating Embedding Recommendation Systems with FlexEMR

arXiv: https://arxiv.org/abs/2410.12794 · HTML: https://mlsanity.com/paper/2410.12794/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 51 references that are in the map.
- First posted on arXiv 28 Sep 2024.

## Builds on (most cited first, 22 of 22)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale](https://mlsanity.com/paper/2201.05596/) (2022)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [The Architectural Implications of Facebook's DNN-based Personalized Recommendation](https://mlsanity.com/paper/1906.03109/) (2019)
- [Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models](https://mlsanity.com/paper/2104.05158/) (2021)
- [Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems](https://mlsanity.com/paper/1909.02107/) (2019)
- [TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models](https://mlsanity.com/paper/2101.11714/) (2021)
- [DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference](https://mlsanity.com/paper/2001.02772/) (2020)
- [RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing](https://mlsanity.com/paper/1912.12953/) (2019)
- [TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learning](https://mlsanity.com/paper/1908.03072/) (2019)
- [Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems](https://mlsanity.com/paper/1909.11810/) (2019)
- [MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Services](https://mlsanity.com/paper/2205.10034/) (2022)
- [RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference](https://mlsanity.com/paper/2102.00075/) (2021)
- [Understanding Capacity-Driven Scale-Out Neural Recommendation Inference](https://mlsanity.com/paper/2011.02084/) (2020)
- [RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation](https://mlsanity.com/paper/2201.10095/) (2022)
- [AutoShard: Automated Embedding Table Sharding for Recommender Systems](https://mlsanity.com/paper/2208.06399/) (2022)
- [DreamShard: Generalizable Embedding Table Placement for Recommender Systems](https://mlsanity.com/paper/2210.02023/) (2022)
- [A GPU-specialized Inference Parameter Server for Large-Scale Deep Recommendation Models](https://mlsanity.com/paper/2210.08804/) (2022)
- [MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendation](https://mlsanity.com/paper/2302.10872/) (2023)
- [DisaggRec: Architecting Disaggregated Systems for Large-Scale Personalized Recommendation](https://mlsanity.com/paper/2212.00939/) (2022)
- [FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Models](https://mlsanity.com/paper/2301.02959/) (2023)
- [DxPU: Large Scale Disaggregated GPU Pools in the Datacenter](https://mlsanity.com/paper/2310.04648/) (2023)

## Built on by (most cited first, 1 of 1)

- [Tiara: A Programmable Line-Rate ISA for Remote Memory Access](https://mlsanity.com/paper/2606.13708/) (2026)

## Abstract (arXiv)

Efficiently serving embedding-based recommendation (EMR) models remains a significant challenge due to their increasingly large memory requirements. Today's practice splits the model across many monolithic servers, where a mix of GPUs, CPUs, and DRAM is provisioned in fixed proportions. This approach leads to suboptimal resource utilization and increased costs. Disaggregating embedding operations from neural network inference is a promising solution but raises novel networking challenges. In this paper, we discuss the design of FlexEMR for optimized EMR disaggregation. FlexEMR proposes two sets of techniques to tackle the networking challenges: Leveraging the temporal and spatial locality of embedding lookups to reduce data movement over the network, and designing an optimized multi-threaded RDMA engine for concurrent lookup subrequests. We outline the design space for each technique and present initial results from our early prototype.