# End-to-End User Behavior Retrieval in Click-Through RatePrediction Model

arXiv: https://arxiv.org/abs/2108.04468 · HTML: https://mlsanity.com/paper/2108.04468/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 10 Aug 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://mlsanity.com/paper/1803.05170/) (2018)
- [Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction](https://mlsanity.com/paper/2006.05639/) (2020)
- [Product-based Neural Networks for User Response Prediction](https://mlsanity.com/paper/1611.00144/) (2016)
- [Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks](https://mlsanity.com/paper/1708.04617/) (2017)
- [A Simple Convolutional Generative Network for Next Item Recommendation](https://mlsanity.com/paper/1808.05163/) (2018)
- [Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.09248/) (2019)
- [Multi-Interest Network with Dynamic Routing for Recommendation at Tmall](https://mlsanity.com/paper/1904.08030/) (2019)
- [Practical and Optimal LSH for Angular Distance](https://mlsanity.com/paper/1509.02897/) (2015)
- [Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction](https://mlsanity.com/paper/1601.02376/) (2016)
- [Deep Session Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.06482/) (2019)
- [Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising](https://mlsanity.com/paper/1806.03514/) (2018)
- [User Behavior Retrieval for Click-Through Rate Prediction](https://mlsanity.com/paper/2005.14171/) (2020)
- [Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction](https://mlsanity.com/paper/1905.00758/) (2019)
- [Joint Optimization of Tree-based Index and Deep Model for Recommender Systems](https://mlsanity.com/paper/1902.07565/) (2019)
- [Deep Interest with Hierarchical Attention Network for Click-Through Rate Prediction](https://mlsanity.com/paper/2005.12981/) (2020)

## Built on by (most cited first, 30 of 45)

- [Integrating Summarization and Retrieval for Enhanced Personalization via Large Language Models](https://mlsanity.com/paper/2310.20081/) (2023)
- [TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou](https://mlsanity.com/paper/2407.16357/) (2024)
- [A Survey on User Behavior Modeling in Recommender Systems](https://mlsanity.com/paper/2302.11087/) (2023)
- [Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction](https://mlsanity.com/paper/2205.10249/) (2022)
- [Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspective](https://mlsanity.com/paper/2402.02820/) (2024)
- [Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation](https://mlsanity.com/paper/2408.03533/) (2024)
- [Temporal Interest Network for User Response Prediction](https://mlsanity.com/paper/2308.08487/) (2023)
- [PIER: Permutation-Level Interest-Based End-to-End Re-ranking Framework in E-commerce](https://mlsanity.com/paper/2302.03487/) (2023)
- [Long-Sequence Recommendation Models Need Decoupled Embeddings](https://mlsanity.com/paper/2410.02604/) (2024)
- [MUSE: A Simple Yet Effective Multimodal Search-Based Framework for Lifelong User Interest Modeling](https://mlsanity.com/paper/2512.07216/) (2025)
- [BASM: A Bottom-up Adaptive Spatiotemporal Model for Online Food Ordering Service](https://mlsanity.com/paper/2211.12033/) (2022)
- [NLGR: Utilizing Neighbor Lists for Generative Rerank in Personalized Recommendation Systems](https://mlsanity.com/paper/2502.06097/) (2025)
- [Diffusion-based Multi-modal Synergy Interest Network for Click-through Rate Prediction](https://mlsanity.com/paper/2508.21460/) (2025)
- [HyTRec: A Hybrid Temporal-Aware Attention Architecture for Long Behavior Sequential Recommendation](https://mlsanity.com/paper/2602.18283/) (2026)
- [ESMC: Entire Space Multi-Task Model for Post-Click Conversion Rate via Parameter Constraint](https://mlsanity.com/paper/2307.09193/) (2023)
- [ConvFormer: Revisiting Transformer for Sequential User Modeling](https://mlsanity.com/paper/2308.02925/) (2023)
- [TBIN: Modeling Long Textual Behavior Data for CTR Prediction](https://mlsanity.com/paper/2308.08483/) (2023)
- [Fragment and Integrate Network (FIN): A Novel Spatial-Temporal Modeling Based on Long Sequential Behavior for Online Food Ordering Click-Through Rate Prediction](https://mlsanity.com/paper/2308.15703/) (2023)
- [SessionRec: Next Session Prediction Paradigm For Generative Sequential Recommendation](https://mlsanity.com/paper/2502.10157/) (2025)
- [Action is All You Need: Dual-Flow Generative Ranking Network for Recommendation](https://mlsanity.com/paper/2505.16752/) (2025)
- [GR-LLMs: Recent Advances in Generative Recommendation Based on Large Language Models](https://mlsanity.com/paper/2507.06507/) (2025)
- [LLM-Aligned Geographic Item Tokenization for Local-Life Recommendation](https://mlsanity.com/paper/2511.14221/) (2025)
- [AutoAttention: Automatic Field Pair Selection for Attention in User Behavior Modeling](https://mlsanity.com/paper/2210.15154/) (2022)
- [A Preference-oriented Diversity Model Based on Mutual-information in Re-ranking for E-commerce Search](https://mlsanity.com/paper/2405.15521/) (2024)
- [Pre-train and Fine-tune: Recommenders as Large Models](https://mlsanity.com/paper/2501.14268/) (2025)
- [Equip Pre-ranking with Target Attention by Residual Quantization](https://mlsanity.com/paper/2509.16931/) (2025)
- [RelayGR: Scaling Long-Sequence Generative Recommendation via Cross-Stage Relay-Race Inference](https://mlsanity.com/paper/2601.01712/) (2026)
- [Beyond Interleaving: Causal Attention Reformulations for Generative Recommender Systems](https://mlsanity.com/paper/2603.10369/) (2026)
- [Retrieval-Oriented Knowledge for Click-Through Rate Prediction](https://mlsanity.com/paper/2404.18304/) (2024)
- [Generative Click-through Rate Prediction with Applications to Search Advertising](https://mlsanity.com/paper/2507.11246/) (2025)

## Abstract (arXiv)

Click-Through Rate (CTR) prediction is one of the core tasks in recommender systems (RS). It predicts a personalized click probability for each user-item pair. Recently, researchers have found that the performance of CTR model can be improved greatly by taking user behavior sequence into consideration, especially long-term user behavior sequence. The report on an e-commerce website shows that 23% of users have more than 1000 clicks during the past 5 months. Though there are numerous works focus on modeling sequential user behaviors, few works can handle long-term user behavior sequence due to the strict inference time constraint in real world system. Two-stage methods are proposed to push the limit for better performance. At the first stage, an auxiliary task is designed to retrieve the top-k similar items from long-term user behavior sequence. At the second stage, the classical attention mechanism is conducted between the candidate item and k items selected in the first stage. However, information gap happens between retrieval stage and the main CTR task. This goal divergence can greatly diminishing the performance gain of long-term user sequence. In this paper, inspired by Reformer, we propose a locality-sensitive hashing (LSH) method called ETA (End-to-end Target Attention) which can greatly reduce the training and inference cost and make the end-to-end training with long-term user behavior sequence possible. Both offline and online experiments confirm the effectiveness of our model. We deploy ETA into a large-scale real world E-commerce system and achieve extra 3.1% improvements on GMV (Gross Merchandise Value) compared to a two-stage long user sequence CTR model.