mlsanityData to 14 Oct 2026

Wukong: Towards a Scaling Law for Large-Scale Recommendation

arXiv 2403.02545 ↗First posted 4 Mar 2024cs.LG cs.AIcited by 74

Not on the map. Why?

Abstract (arXiv)

Scaling laws play an instrumental role in the sustainable improvement in model quality. Unfortunately, recommendation models to date do not exhibit such laws similar to those observed in the domain of large language models, due to the inefficiencies of their upscaling mechanisms. This limitation poses significant challenges in adapting these models to increasingly more complex real-world datasets. In this paper, we propose an effective network architecture based purely on stacked factorization machines, and a synergistic upscaling strategy, collectively dubbed Wukong, to establish a scaling law in the domain of recommendation. Wukong's unique design makes it possible to capture diverse, any-order of interactions simply through taller and wider layers. We conducted extensive evaluations on six public datasets, and our results demonstrate that Wukong consistently outperforms state-of-the-art models quality-wise. Further, we assessed Wukong's scalability on an internal, large-scale dataset. The results show that Wukong retains its superiority in quality over state-of-the-art models, while holding the scaling law across two orders of magnitude in model complexity, extending beyond 100 GFLOP/example, where prior arts fall short.

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

34 of 50 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2016Layer Normalizationcited by 5,669
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 676
  7. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  8. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  9. 2020DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systemscited by 327
  10. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 320
  11. 2018xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systemscited by 278
  12. 2024Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendationscited by 188
22 more
  1. 2022BARS: Towards Open Benchmarking for Recommender Systemscited by 54
  2. 2020Open Benchmarking for Click-Through Rate Predictioncited by 53
  3. 2019Adaptive Factorization Network: Learning Adaptive-Order Feature Interactionscited by 48
  4. 2021MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Maskcited by 48
  5. 2023VIP5: Towards Multimodal Foundation Models for Recommendationcited by 46
  6. 2016Higher-Order Factorization Machinescited by 45
  7. 2021Torch.fx: Practical Program Capture and Transformation for Deep Learning in Pythoncited by 40
  8. 2020Learning to Embed Categorical Features without Embedding Tables for Recommendationcited by 38
  9. 2022Understanding Scaling Laws for Recommendation Modelscited by 38
  10. 2023FinalMLP: An Enhanced Two-Stream MLP Model for CTR Predictioncited by 37
  11. 2022Monolith: Real Time Recommendation System With Collisionless Embedding Tablecited by 30
  12. 2023On the Embedding Collapse when Scaling up Recommendation Modelscited by 29
  13. 2021Scaling Law for Recommendation Models: Towards General-purpose User Representationscited by 26
  14. 2023Scaling Law of Large Sequential Recommendation Modelscited by 23
  15. 2018Parameter Hub: a Rack-Scale Parameter Server for Distributed Deep Neural Network Trainingcited by 17
  16. 2021Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameterscited by 17
  17. 2023Hiformer: Heterogeneous Feature Interactions Learning with Transformers for Recommender Systemscited by 14
  18. 2023Improving Training Stability for Multitask Ranking Models in Recommender Systemscited by 13
  19. 2024Foundation Models for Recommender Systems: A Survey and New Perspectivescited by 7
  20. 2024Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large-Scale Recommendationcited by 6
  21. 2023Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Modelscited by 5
  22. 2023Breaking the Curse of Quality Saturation with User-Centric Rankingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 22; 2026: 46.

74 papers.

Sort:

By area:

  1. 2024Scaling Optimal LR Across Token Horizonscited by 40
  2. 2024SLMRec: Distilling Large Language Models into Small for Sequential Recommendationcited by 18
  3. 2025MiniOneRec: An Open-Source Framework for Scaling Generative Recommendationcited by 17
  4. 2025External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendationcited by 13
  5. 2026GR2: Generative Reasoning Re-rankercited by 11
  6. 2025The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exitcited by 10
  7. 2024InterFormer: Effective Heterogeneous Interaction Learning for Click-Through Rate Predictioncited by 8
  8. 2026Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Designcited by 8
  9. 2026AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systemscited by 8
  10. 2026UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systemscited by 7
  11. 2025Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamicscited by 5
  12. 2025Yambda-5B -- A Large-Scale Multi-modal Dataset for Ranking And Retrievalcited by 5
38 more
  1. 2025KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Metacited by 5
  2. 2026LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendationcited by 5
  3. 2025Exploring Training and Inference Scaling Laws in Generative Retrievalcited by 4
  4. 2025Large Language Model Empowered Recommendation Meets All-domain Continual Pre-Trainingcited by 4
  5. 2025Field Matters: A Lightweight LLM-enhanced Method for CTR Predictioncited by 4
  6. 2025Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deploymentcited by 4
  7. 2026PACEvolve++: Improving Test-time Learning for Evolutionary Search Agentscited by 4
  8. 2024Scaling Laws for Online Advertisement Retrievalcited by 3
  9. 2025On the Practice of Deep Hierarchical Ensemble Network for Ad Conversion Rate Predictioncited by 3
  10. 2025MTmixAtt: Integrating Mixture-of-Experts with Multi-Mix Attention for Large-Scale Recommendationcited by 3
  11. 2025Revisiting scalable sequential recommendation with Multi-Embedding Approach and Mixture-of-Expertscited by 3
  12. 2025From Scaling to Structured Expressivity: Rethinking Transformers for CTR Predictioncited by 3
  13. 2026TokenFormer: Unify the Multi-Field and Sequential Recommendation Worldscited by 3
  14. 2026ReasonRec: A Reasoning-Augmented Multimodal Agent for Unified Recommendationcited by 3
  15. 2025BehaveGPT: A Foundation Model for Large-scale User Behavior Modelingcited by 2
  16. 2025RecIS: Sparse to Dense, A Unified Training Framework for Recommendation Modelscited by 2
  17. 2025Adaptive Regularization for Large-Scale Sparse Feature Embedding Modelscited by 2
  18. 2026Hi-SAM: A Hierarchical Structure-Aware Multi-modal Framework for Large-Scale Recommendationcited by 2
  19. 2026SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scalingcited by 2
  20. 2026UniMoMo: Expert Merging-Based MoE Acceleration for Large Recommendation Modelscited by 2
  21. 2024CubicML: Automated ML for Large ML Systems Co-design with ML Prediction of Performancecited by 1
  22. 2024Enhancing Performance and Scalability of Large-Scale Recommendation Systems with Jagged Flash Attentioncited by 1
  23. 2025Privacy Preserving Conversion Modeling in Data Clean Roomcited by 1
  24. 2025asLLR: LLM based Leads Ranking in Auto Salescited by 1
  25. 2026Compute Only Once: UG-Separation for Efficient Large Recommendation Modelscited by 1
  26. 2026Closing the Auto-Research Loop: An AI Co-Scientist for Production Search Rankingcited by 1
  27. 2026LoKA: Low-precision Kernel Applications for Recommendation Models At Scalecited by 1
  28. 2026Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendationcited by 1
  29. 2026SlimPer: Make Personalization Model Slim and Smartcited by 1
  30. 2025From Features to Transformers: Redefining Ranking for Scalable Impactcited by 0
  31. 2025Phantora: Maximizing Code Reuse in Simulation-based Machine Learning System Performance Estimationcited by 0
  32. 2025DynamiX: Dynamic Resource eXploration for Personalized Ad-Recommendationscited by 0
  33. 2026GRAB: An LLM-Inspired Sequence-First Click-Through Rate Prediction Modeling Paradigmcited by 0
  34. 2026MDL: A Unified Multi-Distribution Learner in Large-scale Industrial Recommendation through Tokenizationcited by 0
  35. 2026ML-DCN: Masked Low-Rank Deep Crossing Network Towards Scalable Ads Click-through Rate Prediction at Pinterestcited by 0
  36. 2026Divine Benevolence is an x2: GLUs scale asymptotically faster than MLPscited by 0
  37. 2026SOLAR: SVD-Optimized Lifelong Attention for Recommendationcited by 0
  38. 2026Feed m Birds with One Scone: Accelerating Multi-task Gradient Balancing via Bi-level Optimizationcited by 0

50 most cited of 74 All 74 (JSON)