mlsanityData to 7 Oct 2026

DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference

arXiv 2001.02772 ↗First posted 8 Jan 2020cs.DCcited by 54

Not on the map. Why?

Abstract (arXiv)

Neural personalized recommendation is the corner-stone of a wide collection of cloud services and products, constituting significant compute demand of the cloud infrastructure. Thus, improving the execution efficiency of neural recommendation directly translates into infrastructure capacity saving. In this paper, we devise a novel end-to-end modeling infrastructure, DeepRecInfra, that adopts an algorithm and system co-design methodology to custom-design systems for recommendation use cases. Leveraging the insights from the recommendation characterization, a new dynamic scheduler, DeepRecSched, is proposed to maximize latency-bounded throughput by taking into account characteristics of inference query size and arrival patterns, recommendation model architectures, and underlying hardware systems. By doing so, system throughput is doubled across the eight industry-representative recommendation models. Finally, design, deployment, and evaluation in at-scale production datacenter shows over 30% latency reduction across a wide variety of recommendation models running on hundreds of machines.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 13; 2023: 12; 2024: 12; 2025: 4; 2026: 1.

54 papers.

Sort:

By area:

  1. 2023S3: Increasing GPU Utilization during Generative Inference for Higher Throughputcited by 81
  2. 2021TT-Rec: Tensor Train Compression for Deep Learning Recommendation Modelscited by 58
  3. 2024Andes: Defining and Enhancing Quality-of-Experience in LLM-Based Text Streaming Servicescited by 44
  4. 2021RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inferencecited by 35
  5. 2022A Comprehensive Survey on Trustworthy Recommender Systemscited by 32
  6. 2021Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Trainingcited by 28
  7. 2022RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendationcited by 28
  8. 2022Merlin HugeCTR: GPU-accelerated Recommender System Training and Inferencecited by 20
  9. 2021MAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Corescited by 15
  10. 2023ERASER: Machine Unlearning in MLaaS via an Inference Serving-Aware Approachcited by 15
  11. 2023MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systemscited by 14
  12. 2022Heterogeneous Acceleration Pipeline for Recommendation System Trainingcited by 13
38 more
  1. 2022Training Personalized Recommendation Systems from (GPU) Scratch: Look Forward not Backwardscited by 11
  2. 2022BagPipe: Accelerating Deep Recommendation Model Trainingcited by 10
  3. 2022A GPU-specialized Inference Parameter Server for Large-Scale Deep Recommendation Modelscited by 9
  4. 2023MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendationcited by 9
  5. 2021On Estimating Recommendation Evaluation Metrics under Samplingcited by 8
  6. 2021RecPipe: Co-designing Models and Hardware to Jointly Optimize Recommendation Quality and Performancecited by 8
  7. 2021Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scalecited by 7
  8. 2023CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Modelscited by 7
  9. 2021Differentiable NAS Framework and Application to Ads CTR Predictioncited by 6
  10. 2022Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Visioncited by 6
  11. 2022Learning to Collide: Recommendation System Model Compression with Learned Hash Functionscited by 5
  12. 2023GPU-based Private Information Retrieval for On-Device Machine Learning Inferencecited by 5
  13. 2025Request-Only Optimization for Recommendation Systemscited by 5
  14. 2023FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Modelscited by 4
  15. 2021Demystifying BERT: Implications for Accelerator Designcited by 3
  16. 2021Alternate Model Growth and Pruning for Efficient Training of Recommendation Systemscited by 3
  17. 2022KAIROS: Building Cost-Efficient Machine Learning Inference Systems with Heterogeneous Cloud Resourcescited by 3
  18. 2023Mem-Rec: Memory Efficient Recommendation System using Alternative Representationcited by 3
  19. 2023ACCL+: an FPGA-Based Collective Engine for Distributed Applicationscited by 3
  20. 2024AnnotatedTables: A Large Tabular Dataset with Language Model Annotationscited by 3
  21. 2022PARIS and ELSA: An Elastic Scheduling Algorithm for Reconfigurable Multi-GPU Inference Serverscited by 2
  22. 2024PreSto: An In-Storage Data Preprocessing System for Training Recommendation Modelscited by 2
  23. 2021Cocktail: Leveraging Ensemble Learning for Optimized Model Serving in Public Cloudcited by 1
  24. 2022RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instancescited by 1
  25. 2023Failure Tolerant Training with Persistent Memory Disaggregation over CXLcited by 1
  26. 2024LazyDP: Co-Designing Algorithm-Software for Scalable Training of Differentially Private Recommendation Modelscited by 1
  27. 2024CADC: Encoding User-Item Interactions for Compressing Recommendation Model Training Datacited by 1
  28. 2024PIFS-Rec: Process-In-Fabric-Switch for Large-Scale Recommendation System Inferencescited by 1
  29. 2024Disaggregating Embedding Recommendation Systems with FlexEMRcited by 1
  30. 2025Rec-AD: An Efficient Computation Framework for FDIA Detection Based on Tensor Train Decomposition and Deep Learning Recommendation Modelcited by 1
  31. 2025Cocoon: A System Architecture for Differentially Private Training with Correlated Noisescited by 1
  32. 2021ECRM: Efficient Fault Tolerance for Recommendation Model Training via Erasure Codingcited by 0
  33. 2022Dynamic Network Adaptation at Inferencecited by 0
  34. 2023Hera: A Heterogeneity-Aware Multi-Tenant Inference Server for Personalized Recommendationscited by 0
  35. 2023Reclaimer: A Reinforcement Learning Approach to Dynamic Resource Allocation for Cloud Microservicescited by 0
  36. 2024From Data to Decisions: The Transformational Power of Machine Learning in Business Recommendationscited by 0
  37. 2024Carbon Connect: An Ecosystem for Sustainable Computingcited by 0
  38. 2024ElasticRec: A Microservice-based Model Serving Architecture Enabling Elastic Resource Scaling for Recommendation Modelscited by 0

50 most cited of 54 All 54 (JSON)