mlsanityData to 7 Oct 2026

RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation

arXiv 2201.10095 ↗First posted 25 Jan 2022cs.LG cs.AR cs.DCcited by 28

Not on the map. Why?

Abstract (arXiv)

We propose RecShard, a fine-grained embedding table (EMB) partitioning and placement technique for deep learning recommendation models (DLRMs). RecShard is designed based on two key observations. First, not all EMBs are equal, nor all rows within an EMB are equal in terms of access patterns. EMBs exhibit distinct memory characteristics, providing performance optimization opportunities for intelligent EMB partitioning and placement across a tiered memory hierarchy. Second, in modern DLRMs, EMBs function as hash tables. As a result, EMBs display interesting phenomena, such as the birthday paradox, leaving EMBs severely under-utilized. RecShard determines an optimal EMB sharding strategy for a set of EMBs based on training data distributions and model characteristics, along with the bandwidth characteristics of the underlying tiered memory hierarchy. In doing so, RecShard achieves over 6 times higher EMB training throughput on average for capacity constrained DLRMs. The throughput increase comes from improved EMB load balance by over 12 times and from the reduced access to the slower memory by over 87 times.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 30 of its 53 references that are in the map. First posted on arXiv 25 Jan 2022.

Builds on

30 of 53 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2016Wide & Deep Learning for Recommender Systemscited by 745
  5. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  6. 2018Deep Interest Evolution Network for Click-Through Rate Predictioncited by 336
  7. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  8. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  9. 2019MLPerf Inference Benchmarkcited by 195
  10. 2019MLPerf Training Benchmarkcited by 114
  11. 2020Improving data-driven global weather prediction using deep convolutional neural networks on a cubed spherecited by 95
  12. 2019The Architectural Implications of Facebook's DNN-based Personalized Recommendationcited by 83
18 more
  1. 2020Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systemscited by 59
  2. 2021TT-Rec: Tensor Train Compression for Deep Learning Recommendation Modelscited by 58
  3. 2020DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inferencecited by 54
  4. 2019RecNMP: Accelerating Personalized Recommendation with Near-Memory Processingcited by 49
  5. 2020Understanding Training Efficiency of Deep Learning Recommendation Models at Scalecited by 43
  6. 2019Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systemscited by 41
  7. 2019Neural Input Search for Large Scale Recommendation Modelscited by 39
  8. 2020Learning to Embed Categorical Features without Embedding Tables for Recommendationcited by 38
  9. 2020Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systemscited by 37
  10. 2021RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inferencecited by 35
  11. 2020Model Size Reduction Using Frequency Based Double Hashing for Recommender Systemscited by 33
  12. 2020Understanding Capacity-Driven Scale-Out Neural Recommendation Inferencecited by 29
  13. 2020Developing a Recommendation Benchmark for MLPerf Training and Inferencecited by 12
  14. 2020Cross-Stack Workload Characterization of Deep Recommendation Systemscited by 11
  15. 2020CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recoverycited by 11
  16. 2019Exploiting Parallelism Opportunities with Deep Learning Frameworkscited by 10
  17. 2020EMOGI: Efficient Memory-access for Out-of-memory Graph-traversal In GPUscited by 10
  18. 2020Exploring the limits of Concurrency in ML Training on Google TPUscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 7; 2024: 10; 2025: 3; 2026: 1.

28 papers.

Sort:

  1. 2022A Comprehensive Survey on Trustworthy Recommender Systemscited by 32
  2. 2022SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problemscited by 21
  3. 2022AutoShard: Automated Embedding Table Sharding for Recommender Systemscited by 17
  4. 2022DreamShard: Generalizable Embedding Table Placement for Recommender Systemscited by 14
  5. 2023MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systemscited by 14
  6. 2022Heterogeneous Acceleration Pipeline for Recommendation System Trainingcited by 13
  7. 2023Experimental Analysis of Large-scale Learnable Vector Storage Compressioncited by 12
  8. 2022RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructurecited by 9
  9. 2023MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendationcited by 9
  10. 2023Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Modelscited by 5
  11. 2024cedar: Optimized and Unified Machine Learning Input Data Pipelinescited by 5
  12. 2023FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Modelscited by 4
16 more
  1. 2023InTune: Reinforcement Learning-based Data Pipeline Optimization for Deep Recommendation Modelscited by 3
  2. 2022Data Leakage via Access Patterns of Sparse Features in Deep Learning-based Recommendation Systemscited by 1
  3. 2024Fine-Grained Embedding Dimension Optimization During Training for Recommender Systemscited by 1
  4. 2024Towards Universal Performance Modeling for Machine Learning Training on Multi-GPU Platformscited by 1
  5. 2024CADC: Encoding User-Item Interactions for Compressing Recommendation Model Training Datacited by 1
  6. 2024An Enhanced Batch Query Architecture in Real-time Recommendationcited by 1
  7. 2024PIFS-Rec: Process-In-Fabric-Switch for Large-Scale Recommendation System Inferencescited by 1
  8. 2024Disaggregating Embedding Recommendation Systems with FlexEMRcited by 1
  9. 2025Near-Zero-Overhead Freshness for Recommendation Systems via Inference-Side Model Updatescited by 1
  10. 2023Review of compressed embedding layers and their applications for recommender systemscited by 0
  11. 2024ProactivePIM: Accelerating Weight-Sharing Embedding Layer with PIM for Scalable Recommendation Systemcited by 0
  12. 2024ElasticRec: A Microservice-based Model Serving Architecture Enabling Elastic Resource Scaling for Recommendation Modelscited by 0
  13. 2024Pushing the Performance Envelope of DNN-based Recommendation Systems Inference on GPUscited by 0
  14. 2025SCRec: A Scalable Computational Storage System with Statistical Sharding and Tensor-train Decomposition for Recommendation Modelscited by 0
  15. 2025Ember: A Compiler for Efficient Embedding Operations on Decoupled Access-Execute Architecturescited by 0
  16. 2026SURGE: SuperBatch Unified Resource-efficient GPU Encoding for Heterogeneous Partitioned Datacited by 0