mlsanityData to 14 Oct 2026

Disaggregating Embedding Recommendation Systems with FlexEMR

arXiv 2410.12794 ↗First posted 28 Sep 2024cs.IR cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Efficiently serving embedding-based recommendation (EMR) models remains a significant challenge due to their increasingly large memory requirements. Today's practice splits the model across many monolithic servers, where a mix of GPUs, CPUs, and DRAM is provisioned in fixed proportions. This approach leads to suboptimal resource utilization and increased costs. Disaggregating embedding operations from neural network inference is a promising solution but raises novel networking challenges. In this paper, we discuss the design of FlexEMR for optimized EMR disaggregation. FlexEMR proposes two sets of techniques to tackle the networking challenges: Leveraging the temporal and spatial locality of embedding lookups to reduce data movement over the network, and designing an optimized multi-threaded RDMA engine for concurrent lookup subrequests. We outline the design space for each technique and present initial results from our early prototype.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 51 references that are in the map. First posted on arXiv 28 Sep 2024.

Builds on

22 of 51 references.

References · 22

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 404
  3. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  4. 2019The Architectural Implications of Facebook's DNN-based Personalized Recommendationcited by 83
  5. 2021Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Modelscited by 78
  6. 2019Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systemscited by 68
  7. 2021TT-Rec: Tensor Train Compression for Deep Learning Recommendation Modelscited by 58
  8. 2020DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inferencecited by 54
  9. 2019RecNMP: Accelerating Personalized Recommendation with Near-Memory Processingcited by 49
  10. 2019TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learningcited by 43
  11. 2019Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systemscited by 41
  12. 2022MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Servicescited by 37
10 more
  1. 2021RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inferencecited by 35
  2. 2020Understanding Capacity-Driven Scale-Out Neural Recommendation Inferencecited by 29
  3. 2022RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendationcited by 28
  4. 2022AutoShard: Automated Embedding Table Sharding for Recommender Systemscited by 17
  5. 2022DreamShard: Generalizable Embedding Table Placement for Recommender Systemscited by 14
  6. 2022A GPU-specialized Inference Parameter Server for Large-Scale Deep Recommendation Modelscited by 9
  7. 2023MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendationcited by 9
  8. 2022DisaggRec: Architecting Disaggregated Systems for Large-Scale Personalized Recommendationcited by 5
  9. 2023FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Modelscited by 4
  10. 2023DxPU: Large Scale Disaggregated GPU Pools in the Datacentercited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Tiara: A Programmable Line-Rate ISA for Remote Memory Accesscited by 0