# Self-Evolving Memory for Generative Recommendation

arXiv: https://arxiv.org/abs/2609.15598 · HTML: https://mlsanity.com/paper/2609.15598/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 37 references that are in the map.
- First posted on arXiv 14 Sep 2026.

## Builds on (most cited first, 27 of 27)

- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models](https://mlsanity.com/paper/2401.06066/) (2024)
- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [MemoryBank: Enhancing Large Language Models with Long-Term Memory](https://mlsanity.com/paper/2305.10250/) (2023)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [Recommender Systems with Generative Retrieval](https://mlsanity.com/paper/2305.05065/) (2023)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment](https://mlsanity.com/paper/2502.18965/) (2025)
- [Learnable Item Tokenization for Generative Recommendation](https://mlsanity.com/paper/2405.07314/) (2024)
- [MTGR: Industrial-Scale Generative Recommendation Framework in Meituan](https://mlsanity.com/paper/2505.18654/) (2025)
- [PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations](https://mlsanity.com/paper/2510.07784/) (2025)
- [Federated Adaptation for Foundation Model-based Recommendations](https://mlsanity.com/paper/2405.04840/) (2024)
- [Structure Aware Incremental Learning with Personalized Imitation Weights for Recommender Systems](https://mlsanity.com/paper/2305.01204/) (2023)
- [Memory Assisted LLM for Personalized Recommendation System](https://mlsanity.com/paper/2505.03824/) (2025)
- [Preliminary Study on Incremental Learning for Large Language Model-based Recommender Systems](https://mlsanity.com/paper/2312.15599/) (2023)
- [Continual Collaborative Distillation for Recommender System](https://mlsanity.com/paper/2405.19046/) (2024)
- [Leveraging Memory Retrieval to Enhance LLM-based Generative Recommendation](https://mlsanity.com/paper/2412.17593/) (2024)
- [Autonomous Information Seeking: A Roadmap for Agentic Recommender Systems](https://mlsanity.com/paper/2607.04433/) (2026)
- [Recurrent Preference Memory for Efficient Long-Sequence Generative Recommendation](https://mlsanity.com/paper/2602.11605/) (2026)
- [Continual Recommender Systems](https://mlsanity.com/paper/2507.03861/) (2025)
- [Temporal User Profiling with LLMs: Balancing Short-Term and Long-Term Preferences for Recommendations](https://mlsanity.com/paper/2508.08454/) (2025)
- [Continual Low-Rank Adapters for LLM-based Generative Recommender Systems](https://mlsanity.com/paper/2510.25093/) (2025)
- [Drift-Aware Continual Tokenization for Generative Recommendation](https://mlsanity.com/paper/2603.29705/) (2026)
- [MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches](https://mlsanity.com/paper/2604.22881/) (2026)
- [Effectiveness of LLMs in Temporal User Profiling for Recommendation](https://mlsanity.com/paper/2511.00176/) (2025)

## Abstract (arXiv)

Generative recommendation has emerged as a promising end-to-end paradigm for personalized recommendation. However, user preferences continuously evolve over time, making self-evolving an essential capability for generative recommender systems. Existing evolving strategies, such as continual retraining and distillation-based adaptation, directly update the shared model parameters using streaming interactions. Nevertheless, we find that directly applying such strategies to generative recommendation introduces a critical issue, termed evolution conflict. Specifically, heterogeneous preference shifts from different users are optimized within a fully shared autoregressive parameter space, causing dominant behavioral patterns to progressively dominate the model evolution process while underrepresented patterns become increasingly overlooked. To address this issue, we propose a self-evolving memory paradigm for generative recommendation, aiming to enable effective evolution across heterogeneous behavioral patterns. We further identify three key principles for effective self-evolving recommendation systems, including isolated memorization, reinforced evolution, and scalable application. Guided by these principles, we develop LION, a simple yet effective framework centered on a sparse Key-Value memory layer. Specifically, LION introduces sparse memory activation to isolate the evolution of different behavioral patterns, while a consolidation loss is designed to reinforce the learning of underrepresented preference dynamics during continual adaptation. Extensive experiments on diverse real-world datasets demonstrate the effectiveness of LION under various continual evolution settings (e.g., per-period evaluation, user/item group evaluation, and evolution convergence analysis). The codes are released at https://github.com/JazyJiang/Self-Evolving-Memory-for-Generative-Recommendation.