# Scaling Law of Large Sequential Recommendation Models

arXiv: https://arxiv.org/abs/2311.11351 · HTML: https://mlsanity.com/paper/2311.11351/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2023.

## Built on by (most cited first, 23 of 23)

- [Wukong: Towards a Scaling Law for Large-Scale Recommendation](https://mlsanity.com/paper/2403.02545/) (2024)
- [Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations](https://mlsanity.com/paper/2503.02453/) (2025)
- [SLMRec: Distilling Large Language Models into Small for Sequential Recommendation](https://mlsanity.com/paper/2405.17890/) (2024)
- [PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform](https://mlsanity.com/paper/2507.12704/) (2025)
- [Generative Recommendation for Large-Scale Advertising](https://mlsanity.com/paper/2602.22732/) (2026)
- [Understanding Generative Recommendation with Semantic IDs from a Model-scaling View](https://mlsanity.com/paper/2509.25522/) (2025)
- [Yambda-5B -- A Large-Scale Multi-modal Dataset for Ranking And Retrieval](https://mlsanity.com/paper/2505.22238/) (2025)
- [TBGRecall: A Generative Retrieval Model for E-commerce Recommendation Scenarios](https://mlsanity.com/paper/2508.11977/) (2025)
- [Exploring Training and Inference Scaling Laws in Generative Retrieval](https://mlsanity.com/paper/2503.18941/) (2025)
- [Large Language Model Empowered Recommendation Meets All-domain Continual Pre-Training](https://mlsanity.com/paper/2504.08949/) (2025)
- [Scaling Laws for Online Advertisement Retrieval](https://mlsanity.com/paper/2411.13322/) (2024)
- [Two-dimensional Sparse Parallelism for Large Scale Deep Learning Recommendation Model Training](https://mlsanity.com/paper/2508.03854/) (2025)
- [Hi-SAM: A Hierarchical Structure-Aware Multi-modal Framework for Large-Scale Recommendation](https://mlsanity.com/paper/2602.11799/) (2026)
- [Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation](https://mlsanity.com/paper/2605.23191/) (2026)
- [SAGE: Scale-Aware Gradual Evolution for Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2508.11347/) (2025)
- [Closing the Performance Gap in Generative Recommenders with Collaborative Tokenization and Efficient Modeling](https://mlsanity.com/paper/2508.14910/) (2025)
- [GRAB: An LLM-Inspired Sequence-First Click-Through Rate Prediction Modeling Paradigm](https://mlsanity.com/paper/2602.01865/) (2026)
- [NestPipe: Large-Scale Recommendation Training on 1,500+ Accelerators via Nested Pipelining](https://mlsanity.com/paper/2604.06956/) (2026)
- [DUET -- Dual User Embedding Transformers for Offsite Conversion Prediction](https://mlsanity.com/paper/2606.10243/) (2026)
- [The Pitfall of Scaling Up: Uncovering and Mitigating Popularity Bias Amplification in Scaling Transformer-based Recommenders](https://mlsanity.com/paper/2606.21911/) (2026)
- [Progressive Alignment of Recommender Foundation Model through Multi-Phase Post-Training](https://mlsanity.com/paper/2608.06792/) (2026)
- [Towards a Densing Law for User Representation Learning at Billion-Scale Capacity](https://mlsanity.com/paper/2608.23392/) (2026)
- [DP-Rec: Towards Dynamic Patching for Efficient Long-Sequence Recommendation](https://mlsanity.com/paper/2609.32215/) (2026)

## Abstract (arXiv)

Scaling of neural networks has recently shown great potential to improve the model capacity in various fields. Specifically, model performance has a power-law relationship with model size or data size, which provides important guidance for the development of large-scale models. However, there is still limited understanding on the scaling effect of user behavior models in recommender systems, where the unique data characteristics (e.g. data scarcity and sparsity) pose new challenges to explore the scaling effect in recommendation tasks. In this work, we focus on investigating the scaling laws in large sequential recommendation models. Specially, we consider a pure ID-based task formulation, where the interaction history of a user is formatted as a chronological sequence of item IDs. We don't incorporate any side information (e.g. item text), because we would like to explore how scaling law holds from the perspective of user behavior. With specially improved strategies, we scale up the model size to 0.8B parameters, making it feasible to explore the scaling effect in a diverse range of model sizes. As the major findings, we empirically show that scaling law still holds for these trained models, even in data-constrained scenarios. We then fit the curve for scaling law, and successfully predict the test loss of the two largest tested model scales. Furthermore, we examine the performance advantage of scaling effect on five challenging recommendation tasks, considering the unique issues (e.g. cold start, robustness, long-term preference) in recommender systems. We find that scaling up the model size can greatly boost the performance on these challenging tasks, which again verifies the benefits of large recommendation models.