# Cross-Stack Workload Characterization of Deep Recommendation Systems

arXiv: https://arxiv.org/abs/2010.05037 · HTML: https://mlsanity.com/paper/2010.05037/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2020.

## Built on by (most cited first, 11 of 11)

- [RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation](https://mlsanity.com/paper/2201.10095/) (2022)
- [MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systems](https://mlsanity.com/paper/2310.02784/) (2023)
- [MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendation](https://mlsanity.com/paper/2302.10872/) (2023)
- [RecPipe: Co-designing Models and Hardware to Jointly Optimize Recommendation Quality and Performance](https://mlsanity.com/paper/2105.08820/) (2021)
- [Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale](https://mlsanity.com/paper/2105.12676/) (2021)
- [Mystique: Enabling Accurate and Scalable Generation of Production AI Benchmarks](https://mlsanity.com/paper/2301.04122/) (2022)
- [KAIROS: Building Cost-Efficient Machine Learning Inference Systems with Heterogeneous Cloud Resources](https://mlsanity.com/paper/2210.05889/) (2022)
- [ACCL+: an FPGA-Based Collective Engine for Distributed Applications](https://mlsanity.com/paper/2312.11742/) (2023)
- [RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances](https://mlsanity.com/paper/2207.11434/) (2022)
- [Evaluating and Enhancing Robustness of Deep Recommendation Systems Against Hardware Errors](https://mlsanity.com/paper/2307.10244/) (2023)
- [Pushing the Performance Envelope of DNN-based Recommendation Systems Inference on GPUs](https://mlsanity.com/paper/2410.22249/) (2024)

## Abstract (arXiv)

Deep learning based recommendation systems form the backbone of most personalized cloud services. Though the computer architecture community has recently started to take notice of deep recommendation inference, the resulting solutions have taken wildly different approaches - ranging from near memory processing to at-scale optimizations. To better design future hardware systems for deep recommendation inference, we must first systematically examine and characterize the underlying systems-level impact of design decisions across the different levels of the execution stack. In this paper, we characterize eight industry-representative deep recommendation models at three different levels of the execution stack: algorithms and software, systems platforms, and hardware microarchitectures. Through this cross-stack characterization, we first show that system deployment choices (i.e., CPUs or GPUs, batch size granularity) can give us up to 15x speedup. To better understand the bottlenecks for further optimization, we look at both software operator usage breakdown and CPU frontend and backend microarchitectural inefficiencies. Finally, we model the correlation between key algorithmic model architecture features and hardware bottlenecks, revealing the absence of a single dominant algorithmic component behind each hardware bottleneck.