mlsanityData to 7 Oct 2026

LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation

arXiv 2601.20083 ↗First posted 27 Jan 2026cs.IR cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We present LLaTTE (LLM-Style Latent Transformers for Temporal Events), a scalable transformer architecture for production ads recommendation. Through systematic experiments, we demonstrate that sequence modeling in recommendation systems follows predictable power-law scaling similar to LLMs. Crucially, we find that semantic features bend the scaling curve: they are a prerequisite for scaling, enabling the model to effectively utilize the capacity of deeper and longer architectures. To realize the benefits of continued scaling under strict latency constraints, we introduce a two-stage architecture that offloads the heavy computation of large, long-context models to an asynchronous upstream user model. We demonstrate that upstream improvements transfer predictably to downstream ranking tasks. Deployed as the largest user model at Meta, this multi-stage framework drives a 4.3% conversion uplift on Facebook Feed and Reels with minimal serving overhead, establishing a practical blueprint for harnessing scaling laws in industrial recommender systems.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 32 of its 38 references that are in the map. First posted on arXiv 27 Jan 2026.

Builds on

32 of 38 references.

References · 32

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  3. 2019Root Mean Square Layer Normalizationcited by 1,425
  4. 2018Self-Attentive Sequential Recommendationcited by 1,169
  5. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  6. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  7. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 952
  8. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  9. 2015Session-based Recommendations with Recurrent Neural Networkscited by 685
  10. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 673
  11. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  12. 2017Deep & Cross Network for Ad Click Predictionscited by 392
20 more
  1. 2018Deep Interest Evolution Network for Click-Through Rate Predictioncited by 336
  2. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  3. 2020DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systemscited by 326
  4. 2020S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximizationcited by 311
  5. 2024Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendationscited by 184
  6. 2020Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Predictioncited by 176
  7. 2025OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignmentcited by 167
  8. 2019Behavior Sequence Transformer for E-commerce Recommendation in Alibabacited by 128
  9. 2024Wukong: Towards a Scaling Law for Large-Scale Recommendationcited by 72
  10. 2024HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modelingcited by 63
  11. 2022PinnerFormer: Sequence Modeling for User Representation at Pinterestcited by 57
  12. 2025LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenderscited by 46
  13. 2023TWIN: TWo-stage Interest Network for Lifelong User Behavior Modeling in CTR Prediction at Kuaishoucited by 40
  14. 2022Understanding Scaling Laws for Recommendation Modelscited by 38
  15. 2023TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterestcited by 35
  16. 2025OneTrans: Unified Feature Interaction and Sequence Modeling with One Transformer in Industrial Recommendercited by 34
  17. 2024TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishoucited by 30
  18. 2023Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Metacited by 20
  19. 2024InterFormer: Effective Heterogeneous Interaction Learning for Click-Through Rate Predictioncited by 8
  20. 2025TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systemscited by 7
  2. 2026OCP: Orthogonal Constrained Projection for Sparse Scaling in Industrial Commodity Recommendationcited by 0
  3. 2026LoopFM: Learning frOm HistOrical RePresentations of Foundation Model for Recommendationcited by 0
  4. 2026DUET -- Dual User Embedding Transformers for Offsite Conversion Predictioncited by 0
  5. 2026SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Cachingcited by 0