# Supporting Massive DLRM Inference Through Software Defined Memory

arXiv: https://arxiv.org/abs/2110.11489 · HTML: https://mlsanity.com/paper/2110.11489/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 21 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [The Architectural Implications of Facebook's DNN-based Personalized Recommendation](https://mlsanity.com/paper/1906.03109/) (2019)
- [Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems](https://mlsanity.com/paper/2003.05622/) (2020)
- [FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference](https://mlsanity.com/paper/2101.05615/) (2021)
- [Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems](https://mlsanity.com/paper/2003.09518/) (2020)
- [Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications](https://mlsanity.com/paper/1811.09886/) (2018)
- [RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference](https://mlsanity.com/paper/2102.00075/) (2021)
- [Understanding Capacity-Driven Scale-Out Neural Recommendation Inference](https://mlsanity.com/paper/2011.02084/) (2020)
- [Post-Training 4-bit Quantization on Embedding Tables](https://mlsanity.com/paper/1911.02079/) (2019)
- [Learning from eXtreme Bandit Feedback](https://mlsanity.com/paper/2009.12947/) (2020)
- [Bandana: Using Non-volatile Memory for Storing Deep Learning Models](https://mlsanity.com/paper/1811.05922/) (2018)
- [JIZHI: A Fast and Cost-Effective Model-As-A-Service System for Web-Scale Online Inference at Baidu](https://mlsanity.com/paper/2106.01674/) (2021)
- [First-Generation Inference Accelerator Deployment at Facebook](https://mlsanity.com/paper/2107.04140/) (2021)

## Built on by (most cited first, 5 of 5)

- [Near-Zero-Overhead Freshness for Recommendation Systems via Inference-Side Model Updates](https://mlsanity.com/paper/2512.12295/) (2025)
- [MTrainS: Improving DLRM training efficiency using heterogeneous memories](https://mlsanity.com/paper/2305.01515/) (2023)
- [ProactivePIM: Accelerating Weight-Sharing Embedding Layer with PIM for Scalable Recommendation System](https://mlsanity.com/paper/2402.04032/) (2024)
- [ElasticRec: A Microservice-based Model Serving Architecture Enabling Elastic Resource Scaling for Recommendation Models](https://mlsanity.com/paper/2406.06955/) (2024)
- [Faster Distributed Inference-Only Recommender Systems via Bounded Lag Synchronous Collectives](https://mlsanity.com/paper/2512.19342/) (2025)

## Abstract (arXiv)

Deep Learning Recommendation Models (DLRM) are widespread, account for a considerable data center footprint, and grow by more than 1.5x per year. With model size soon to be in terabytes range, leveraging Storage ClassMemory (SCM) for inference enables lower power consumption and cost. This paper evaluates the major challenges in extending the memory hierarchy to SCM for DLRM, and presents different techniques to improve performance through a Software Defined Memory. We show how underlying technologies such as Nand Flash and 3DXP differentiate, and relate to real world scenarios, enabling from 5% to 29% power savings.