# Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications

arXiv: https://arxiv.org/abs/1811.09886 · HTML: https://mlsanity.com/paper/1811.09886/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2018.

## Built on by (most cited first, 30 of 36)

- [Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models](https://mlsanity.com/paper/2104.05158/) (2021)
- [Compute and Energy Consumption Trends in Deep Learning Inference](https://mlsanity.com/paper/2109.05472/) (2021)
- [RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference](https://mlsanity.com/paper/2102.00075/) (2021)
- [NIPQ: Noise proxy-based Integrated Pseudo-Quantization](https://mlsanity.com/paper/2206.00820/) (2022)
- [Aryl: An Elastic Cluster Scheduler for Deep Learning](https://mlsanity.com/paper/2202.07896/) (2022)
- [A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators](https://mlsanity.com/paper/2105.12842/) (2021)
- [Benchmarking Pathology Foundation Models: Adaptation Strategies and Scenarios](https://mlsanity.com/paper/2410.16038/) (2024)
- [MAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Cores](https://mlsanity.com/paper/2104.13997/) (2021)
- [Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving](https://mlsanity.com/paper/2312.05385/) (2023)
- [SuperServe: Fine-Grained Inference Serving for Unpredictable Workloads](https://mlsanity.com/paper/2312.16733/) (2023)
- [SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learning](https://mlsanity.com/paper/2111.14345/) (2021)
- [JIZHI: A Fast and Cost-Effective Model-As-A-Service System for Web-Scale Online Inference at Baidu](https://mlsanity.com/paper/2106.01674/) (2021)
- [MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendation](https://mlsanity.com/paper/2302.10872/) (2023)
- [Semantically Constrained Memory Allocation (SCMA) for Embedding in Efficient Recommendation Systems](https://mlsanity.com/paper/2103.06124/) (2021)
- [Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale](https://mlsanity.com/paper/2105.12676/) (2021)
- [Benchmarking Resource Usage for Efficient Distributed Deep Learning](https://mlsanity.com/paper/2201.12423/) (2022)
- [Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision](https://mlsanity.com/paper/2205.11913/) (2022)
- [SCAR: Scheduling Multi-Model AI Workloads on Heterogeneous Multi-Chiplet Module Accelerators](https://mlsanity.com/paper/2405.00790/) (2024)
- [Supporting Massive DLRM Inference Through Software Defined Memory](https://mlsanity.com/paper/2110.11489/) (2021)
- [Learning to Collide: Recommendation System Model Compression with Learned Hash Functions](https://mlsanity.com/paper/2203.15837/) (2022)
- [Reconciling High Accuracy, Cost-Efficiency, and Low Latency of Inference Serving Systems](https://mlsanity.com/paper/2304.10892/) (2023)
- [Adaptive Block Floating-Point for Analog Deep Learning Hardware](https://mlsanity.com/paper/2205.06287/) (2022)
- [SMASH: Sparse Matrix Atomic Scratchpad Hashing](https://mlsanity.com/paper/2105.14156/) (2021)
- [Memory Planning for Deep Neural Networks](https://mlsanity.com/paper/2203.00448/) (2022)
- [The trade-offs of model size in large recommendation models : A 10000 × compressed criteo-tb DLRM model (100 GB parameters to mere 10MB)](https://mlsanity.com/paper/2207.10731/) (2022)
- [Demonstrating Analog Inference on the BrainScaleS-2 Mobile System](https://mlsanity.com/paper/2103.15960/) (2021)
- [RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances](https://mlsanity.com/paper/2207.11434/) (2022)
- [Characterizing State Space Model and Hybrid Language Model Performance with Long Context](https://mlsanity.com/paper/2507.12442/) (2025)
- [Dynamic Network Adaptation at Inference](https://mlsanity.com/paper/2204.08400/) (2022)
- [Hera: A Heterogeneity-Aware Multi-Tenant Inference Server for Personalized Recommendations](https://mlsanity.com/paper/2302.11750/) (2023)

## Abstract (arXiv)

The application of deep learning techniques resulted in remarkable improvement of machine learning models. In this paper provides detailed characterizations of deep learning models used in many Facebook social network services. We present computational characteristics of our models, describe high performance optimizations targeting existing systems, point out their limitations and make suggestions for the future general-purpose/accelerated inference hardware. Also, we highlight the need for better co-design of algorithms, numerics and computing platforms to address the challenges of workloads often run in data centers.