mlsanityData to 14 Oct 2026

Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment

arXiv 2508.02929 ↗First posted 4 Aug 2025cs.IR cs.AI cs.LGcited by 4

On the map: Knowledge distillation → Distillation for recommender systems

Abstract (arXiv)

Scaling laws have been established for recommender systems, yet efficiently deploying foundation model (FM) across multiple recommendation surfaces remains a major unsolved challenge. Existing methods for transfer learning face fundamental limitations in this setting: knowledge distillation suffers from transfer fidelity degradation in the large-data regime, and static user or item embeddings lack the expressiveness to capture contextualized user-item interactions. We propose the Foundation-Expert paradigm, where a central FM generates target-aware embeddings which are ingested by lightweight surface-specific expert models. Target-aware embeddings are representations that dynamically capture a user's interest in a specific item conditioned on their full interaction history. Unlike knowledge distillation, which transfers FM knowledge as soft labels, our approach provides these embeddings as input features to each expert model, enabling direct interaction with surface-specific representations. This paradigm achieves transfer ratios of 0.64--1.0 from FM to experts, substantially exceeding existing methods. Fully deployed at Meta serving tens of billions of daily requests since 2025, it delivers 0.050% statistically significant online topline metric improvement and 0.359% cumulative gains across multiple surfaces.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Distillation for recommender systems, one of 12 directions in Knowledge distillation. Builds on 26 of its 41 references that are in the map. First posted on arXiv 4 Aug 2025.

Builds on

26 of 41 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  8. 2020Longformer: The Long-Document Transformercited by 3,413
  9. 2020Knowledge Distillation: A Surveycited by 1,603
  10. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,138
  11. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  12. 2017Learning multiple visual domains with residual adapterscited by 557
14 more
  1. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  2. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
  3. 2024Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendationscited by 188
  4. 2019Practice on Long Sequential User Behavior Modeling for Click-Through Rate Predictioncited by 118
  5. 2023Modular Deep Learningcited by 83
  6. 2024Wukong: Towards a Scaling Law for Large-Scale Recommendationcited by 74
  7. 2025Distillation Scaling Lawscited by 67
  8. 2025MTGR: Industrial-Scale Generative Recommendation Framework in Meituancited by 49
  9. 2025LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenderscited by 47
  10. 2022ItemSage: Learning Product Embeddings for Shopping Recommendations at Pinterestcited by 23
  11. 2023Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Metacited by 21
  12. 2025External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendationcited by 13
  13. 2020General-Purpose User Embeddings based on Mobile App Usagecited by 11
  14. 2024Efficient user history modeling with amortized inference for deep learning recommendation modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms
  2. 2023Embedding in Recommender Systems: A Survey
  3. 2025Revisiting scalable sequential recommendation with Multi-Embedding Approach and Mixture-of-Experts
  4. 2025Improving LLM Interpretability and Performance via Guided Embedding Refinement for Sequential Recommendation
  5. 2025Bridging Collaborative Filtering and Large Language Models with Dynamic Alignment, Multimodal Fusion and Evidence-grounded Explanations
  6. 2023Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations
  7. 2026Inherit4Rec: Parameter Inheritance for Efficient Scaling of Recommendation Models ↗
  8. 2026SMES: Towards Scalable Multi-Task Recommendation via Expert Sparsity