# TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou

arXiv: https://arxiv.org/abs/2407.16357 · HTML: https://mlsanity.com/paper/2407.16357/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 23 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction](https://mlsanity.com/paper/2006.05639/) (2020)
- [Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate](https://mlsanity.com/paper/1804.07931/) (2018)
- [Behavior Sequence Transformer for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1905.06874/) (2019)
- [Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.09248/) (2019)
- [End-to-End User Behavior Retrieval in Click-Through RatePrediction Model](https://mlsanity.com/paper/2108.04468/) (2021)
- [User Behavior Retrieval for Click-Through Rate Prediction](https://mlsanity.com/paper/2005.14171/) (2020)
- [Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction](https://mlsanity.com/paper/1905.00758/) (2019)
- [TWIN: TWo-stage Interest Network for Lifelong User Behavior Modeling in CTR Prediction at Kuaishou](https://mlsanity.com/paper/2302.02352/) (2023)
- [Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction](https://mlsanity.com/paper/2205.10249/) (2022)

## Built on by (most cited first, 30 of 30)

- [Massive Memorization with Hundreds of Trillions of Parameters for Sequential Transducer Generative Recommenders](https://mlsanity.com/paper/2510.22049/) (2025)
- [TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation](https://mlsanity.com/paper/2506.02267/) (2025)
- [PinRec: Unified Generative Retrieval for Pinterest Recommender Systems](https://mlsanity.com/paper/2504.10507/) (2025)
- [Long-Sequence Recommendation Models Need Decoupled Embeddings](https://mlsanity.com/paper/2410.02604/) (2024)
- [LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation](https://mlsanity.com/paper/2601.20083/) (2026)
- [Long-Term Interest Clock: Fine-Grained Time Perception in Streaming Recommendation System](https://mlsanity.com/paper/2501.15817/) (2025)
- [Action is All You Need: Dual-Flow Generative Ranking Network for Recommendation](https://mlsanity.com/paper/2505.16752/) (2025)
- [Equip Pre-ranking with Target Attention by Residual Quantization](https://mlsanity.com/paper/2509.16931/) (2025)
- [RecIS: Sparse to Dense, A Unified Training Framework for Recommendation Models](https://mlsanity.com/paper/2509.20883/) (2025)
- [CaST-POI: Candidate-Conditioned Spatiotemporal Modeling for Next POI Recommendation](https://mlsanity.com/paper/2604.20845/) (2026)
- [ELASTIC: Efficient Linear Attention for Sequential Interest Compression](https://mlsanity.com/paper/2408.09380/) (2024)
- [Efficient Sequential Recommendation for Long Term User Interest Via Personalization](https://mlsanity.com/paper/2601.03479/) (2026)
- [Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning](https://mlsanity.com/paper/2607.14192/) (2026)
- [Full-Stack Optimized Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation](https://mlsanity.com/paper/2501.13344/) (2025)
- [DiffusionGS: Generative Search with Query Conditioned Diffusion in Kuaishou](https://mlsanity.com/paper/2508.17754/) (2025)
- [DMGIN: How Multimodal LLMs Enhance Large Recommendation Models for Lifelong User Post-click Behaviors](https://mlsanity.com/paper/2508.21801/) (2025)
- [GRAB: An LLM-Inspired Sequence-First Click-Through Rate Prediction Modeling Paradigm](https://mlsanity.com/paper/2602.01865/) (2026)
- [SOLAR: SVD-Optimized Lifelong Attention for Recommendation](https://mlsanity.com/paper/2603.02561/) (2026)
- [Versioned Late Materialization for Ultra-Long Sequence Training in Recommendation Systems at Scale](https://mlsanity.com/paper/2604.24806/) (2026)
- [UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence](https://mlsanity.com/paper/2605.09040/) (2026)
- [Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling](https://mlsanity.com/paper/2606.07546/) (2026)
- [MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search](https://mlsanity.com/paper/2607.11030/) (2026)
- [Bumblebee: Interleaved Mixed-Layer Building Blocks for Large-Scale Recommendation Systems](https://mlsanity.com/paper/2607.24804/) (2026)
- [ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation](https://mlsanity.com/paper/2607.27744/) (2026)
- [Interpretable Representation via LLM-Driven Generative Disentanglement for Local-Life Service Recommendation](https://mlsanity.com/paper/2607.27944/) (2026)
- [Decoupled Temporal Encoding for Generative Recommendation](https://mlsanity.com/paper/2608.16274/) (2026)
- [Towards a Densing Law for User Representation Learning at Billion-Scale Capacity](https://mlsanity.com/paper/2608.23392/) (2026)
- [From Feature Interaction to Feature Transport - A Unified Block for Scalable Recommendation Models](https://mlsanity.com/paper/2609.01655/) (2026)
- [SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching](https://mlsanity.com/paper/2609.08443/) (2026)
- [DP-Rec: Towards Dynamic Patching for Efficient Long-Sequence Recommendation](https://mlsanity.com/paper/2609.32215/) (2026)

## Abstract (arXiv)

The significance of modeling long-term user interests for CTR prediction tasks in large-scale recommendation systems is progressively gaining attention among researchers and practitioners. Existing work, such as SIM and TWIN, typically employs a two-stage approach to model long-term user behavior sequences for efficiency concerns. The first stage rapidly retrieves a subset of sequences related to the target item from a long sequence using a search-based mechanism namely the General Search Unit (GSU), while the second stage calculates the interest scores using the Exact Search Unit (ESU) on the retrieved results. Given the extensive length of user behavior sequences spanning the entire life cycle, potentially reaching up to 10^6 in scale, there is currently no effective solution for fully modeling such expansive user interests. To overcome this issue, we introduced TWIN-V2, an enhancement of TWIN, where a divide-and-conquer approach is applied to compress life-cycle behaviors and uncover more accurate and diverse user interests. Specifically, a hierarchical clustering method groups items with similar characteristics in life-cycle behaviors into a single cluster during the offline phase. By limiting the size of clusters, we can compress behavior sequences well beyond the magnitude of 10^5 to a length manageable for online inference in GSU retrieval. Cluster-aware target attention extracts comprehensive and multi-faceted long-term interests of users, thereby making the final recommendation results more accurate and diverse. Extensive offline experiments on a multi-billion-scale industrial dataset and online A/B tests have demonstrated the effectiveness of TWIN-V2. Under an efficient deployment framework, TWIN-V2 has been successfully deployed to the primary traffic that serves hundreds of millions of daily active users at Kuaishou.