mlsanityData to 7 Oct 2026

Understanding Scaling Laws for Recommendation Models

arXiv 2208.08489 ↗First posted 17 Aug 2022cs.IR cs.LGcited by 38

Not on the map. Why?

Abstract (arXiv)

Scale has been a major driving force in improving machine learning performance, and understanding scaling laws is essential for strategic planning for a sustainable model quality performance growth, long-term resource planning and developing efficient system infrastructures to support large-scale models. In this paper, we study empirical scaling laws for DLRM style recommendation models, in particular Click-Through Rate (CTR). We observe that model quality scales with power law plus constant in model size, data size and amount of compute used for training. We characterize scaling efficiency along three different resource dimensions, namely data, parameters and compute by comparing the different scaling schemes along these axes. We show that parameter scaling is out of steam for the model architecture under study, and until a higher-performing model architecture emerges, data scaling is the path forward. The key research questions addressed by this study include: Does a recommendation model scale sustainably as predicted by the scaling laws? Or are we far off from the scaling law predictions? What are the limits of scaling? What are the implications of the scaling laws on long-term hardware/system development?

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 21 of its 33 references that are in the map. First posted on arXiv 17 Aug 2022.

Builds on

21 of 33 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2021Scaling Vision Transformerscited by 993
  4. 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
  5. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  6. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  7. 2021Explaining Neural Scaling Lawscited by 353
  8. 2018Deep Interest Evolution Network for Click-Through Rate Predictioncited by 336
  9. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  10. 2021Scaling Laws for Transfercited by 265
  11. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  12. 2021Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Modelscited by 78
9 more
  1. 2020Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systemscited by 59
  2. 2020Understanding Training Efficiency of Deep Learning Recommendation Models at Scalecited by 43
  3. 2020Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systemscited by 37
  4. 2020Understanding Capacity-Driven Scale-Out Neural Recommendation Inferencecited by 29
  5. 2021Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Trainingcited by 28
  6. 2017Related Pins at Pinterest: The Evolution of a Real-World Recommender Systemcited by 24
  7. 2021Accelerating Recommendation System Training by Leveraging Popular Choicescited by 22
  8. 2019Beyond Human-Level Accuracy: Computational Challenges in Deep Learningcited by 19
  9. 2021Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifierscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 9; 2025: 12; 2026: 10.

38 papers.

Sort:

By area:

  1. 2023The Quantization Model of Neural Scalingcited by 137
  2. 2024Wukong: Towards a Scaling Law for Large-Scale Recommendationcited by 72
  3. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 58
  4. 2024Scaling Laws For Dense Retrievalcited by 28
  5. 2025(Mis)Fitting: A Survey of Scaling Lawscited by 24
  6. 2022Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humanscited by 23
  7. 2025ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilingualitycited by 19
  8. 2024SLMRec: Distilling Large Language Models into Small for Sequential Recommendationcited by 18
  9. 2024Ads Recommendation in a Collapsed and Entangled Worldcited by 13
  10. 2023Improving Training Stability for Multitask Ranking Models in Recommender Systemscited by 12
  11. 2025Model Merging Scaling Laws in Large Language Modelscited by 10
  12. 2022RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructurecited by 9
26 more
  1. 2024Scaling Sequential Recommendation Models with Transformerscited by 7
  2. 2024Exploring the Design Space of Visual Context Representation in Video MLLMscited by 6
  3. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Modelscited by 5
  4. 2026LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendationcited by 5
  5. 2025Exploring Training and Inference Scaling Laws in Generative Retrievalcited by 4
  6. 2025Large Language Model Empowered Recommendation Meets All-domain Continual Pre-Trainingcited by 4
  7. 2025Yambda-5B -- A Large-Scale Multi-modal Dataset for Ranking And Retrievalcited by 4
  8. 2023Ad-Rec: Advanced Feature Interactions to Address Covariate-Shifts in Recommendation Networkscited by 3
  9. 2025xLSTM Scaling Laws: Competitive Performance with Linear Time-Complexitycited by 3
  10. 2026Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendationcited by 3
  11. 2022Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learningcited by 2
  12. 2023Proxy-based Item Representation for Attribute and Context-aware Recommendationcited by 2
  13. 2024Data Efficiency for Large Recommendation Modelscited by 2
  14. 2025Two-dimensional Sparse Parallelism for Large Scale Deep Learning Recommendation Model Trainingcited by 2
  15. 2024Fine-Grained Embedding Dimension Optimization During Training for Recommender Systemscited by 1
  16. 2026COFFEE: COdesign Framework for Feature Enriched Embeddings in Ads-Ranking Systemscited by 1
  17. 2026Hi-SAM: A Hierarchical Structure-Aware Multi-modal Framework for Large-Scale Recommendationcited by 1
  18. 2026LoKA: Low-precision Kernel Applications for Recommendation Models At Scalecited by 1
  19. 2024CubicML: Automated ML for Large ML Systems Co-design with ML Prediction of Performancecited by 0
  20. 2025MLKV: Efficiently Scaling up Large Embedding Model Training with Disk-based Key-Value Storagecited by 0
  21. 2025Exploring Test-time Scaling via Prediction Merging on Large-Scale Recommendationcited by 0
  22. 2026Scaling Laws for Behavioral Foundation Models over User Event Sequencescited by 0
  23. 2026WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecturecited by 0
  24. 2026ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendationcited by 0
  25. 2026Towards a Densing Law for User Representation Learning at Billion-Scale Capacitycited by 0
  26. 2026BAFF: Bid-Aware Filter Family for Mitigating Training Data Interference in RTB A/B Testscited by 0