# Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction

arXiv: https://arxiv.org/abs/1905.00758 · HTML: https://mlsanity.com/paper/1905.00758/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2019.

## Built on by (most cited first, 30 of 41)

- [ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation](https://mlsanity.com/paper/2308.11131/) (2023)
- [End-to-End User Behavior Retrieval in Click-Through RatePrediction Model](https://mlsanity.com/paper/2108.04468/) (2021)
- [Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendation](https://mlsanity.com/paper/2207.05584/) (2022)
- [Bootstrapped Transformer for Offline Reinforcement Learning](https://mlsanity.com/paper/2206.08569/) (2022)
- [Deep Learning for Click-Through Rate Estimation](https://mlsanity.com/paper/2104.10584/) (2021)
- [TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou](https://mlsanity.com/paper/2407.16357/) (2024)
- [A Survey on User Behavior Modeling in Recommender Systems](https://mlsanity.com/paper/2302.11087/) (2023)
- [Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction](https://mlsanity.com/paper/2205.10249/) (2022)
- [Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation](https://mlsanity.com/paper/2408.03533/) (2024)
- [Dynamic Memory based Attention Network for Sequential Recommendation](https://mlsanity.com/paper/2102.09269/) (2021)
- [User Response Prediction in Online Advertising](https://mlsanity.com/paper/2101.02342/) (2021)
- [PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments](https://mlsanity.com/paper/2603.23231/) (2026)
- [Temporal Interest Network for User Response Prediction](https://mlsanity.com/paper/2308.08487/) (2023)
- [DV365: Extremely Long User History Modeling at Instagram](https://mlsanity.com/paper/2506.00450/) (2025)
- [Continuous-Time User Preference Modelling for Temporal Sets Prediction](https://mlsanity.com/paper/2204.05490/) (2022)
- [Learning to Rehearse in Long Sequence Memorization](https://mlsanity.com/paper/2106.01096/) (2021)
- [Empowering General-purpose User Representation with Full-life Cycle Behavior Modeling](https://mlsanity.com/paper/2110.11337/) (2021)
- [Modeling Dynamic User Preference via Dictionary Learning for Sequential Recommendation](https://mlsanity.com/paper/2204.00752/) (2022)
- [MOI-Mixer: Improving MLP-Mixer with Multi Order Interactions in Sequential Recommendation](https://mlsanity.com/paper/2108.07505/) (2021)
- [Efficient Long Sequential User Data Modeling for Click-Through Rate Prediction](https://mlsanity.com/paper/2209.12212/) (2022)
- [ConvFormer: Revisiting Transformer for Sequential User Modeling](https://mlsanity.com/paper/2308.02925/) (2023)
- [TBIN: Modeling Long Textual Behavior Data for CTR Prediction](https://mlsanity.com/paper/2308.08483/) (2023)
- [AutoAttention: Automatic Field Pair Selection for Attention in User Behavior Modeling](https://mlsanity.com/paper/2210.15154/) (2022)
- [Graph Based Long-Term And Short-Term Interest Model for Click-Through Rate Prediction](https://mlsanity.com/paper/2306.10028/) (2023)
- [Learning Complex Users' Preferences for Recommender Systems](https://mlsanity.com/paper/2107.01529/) (2021)
- [MOEF: Modeling Occasion Evolution in Frequency Domain for Promotion-Aware Click-Through Rate Prediction](https://mlsanity.com/paper/2112.13747/) (2021)
- [Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction](https://mlsanity.com/paper/2206.14647/) (2022)
- [USE: Dynamic User Modeling with Stateful Sequence Models](https://mlsanity.com/paper/2403.13344/) (2024)
- [Retrieval and Distill: A Temporal Data Shift-Free Paradigm for Online Recommendation System](https://mlsanity.com/paper/2404.15678/) (2024)
- [Retrieval-Oriented Knowledge for Click-Through Rate Prediction](https://mlsanity.com/paper/2404.18304/) (2024)

## Abstract (arXiv)

User response prediction, which models the user preference w.r.t. the presented items, plays a key role in online services. With two-decade rapid development, nowadays the cumulated user behavior sequences on mature Internet service platforms have become extremely long since the user's first registration. Each user not only has intrinsic tastes, but also keeps changing her personal interests during lifetime. Hence, it is challenging to handle such lifelong sequential modeling for each individual user. Existing methodologies for sequential modeling are only capable of dealing with relatively recent user behaviors, which leaves huge space for modeling long-term especially lifelong sequential patterns to facilitate user modeling. Moreover, one user's behavior may be accounted for various previous behaviors within her whole online activity history, i.e., long-term dependency with multi-scale sequential patterns. In order to tackle these challenges, in this paper, we propose a Hierarchical Periodic Memory Network for lifelong sequential modeling with personalized memorization of sequential patterns for each user. The model also adopts a hierarchical and periodical updating mechanism to capture multi-scale sequential patterns of user interests while supporting the evolving user behavior logs. The experimental results over three large-scale real-world datasets have demonstrated the advantages of our proposed model with significant improvement in user response prediction performance against the state-of-the-arts.