Continual Low-Rank Adapters for LLM-based Generative Recommender Systems
Not on the map. Why?
Abstract (arXiv)
While large language models (LLMs) achieve strong performance in recommendation, they face challenges in continual learning as users, items, and user preferences evolve over time. Existing LoRA-based continual methods primarily focus on preserving performance on previous tasks, but this overlooks the unique nature of recommendation: the goal is not to predict past preferences, and outdated preferences can even harm performance when current interests shift significantly. To address this, we propose PESO (Proximally rEgularized Single evolving lOra, a continual adaptation method for LoRA in recommendation. PESO introduces a proximal regularizer that anchors the current adapter to its most recent frozen state, enabling the model to flexibly balance adaptation and preservation, and to better capture recent user behaviors. Theoretically, we show that this proximal design provides data-aware, direction-wise guidance in the LoRA subspace. Empirically, PESO consistently outperforms existing LoRA-based continual learning methods.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 49 of its 74 references that are in the map. First posted on arXiv 29 Oct 2025.
Builds on
49 of 74 references.
References · 49
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,139
- 2021Learning to Prompt for Continual Learningcited by 944
- 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 337
- 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 312
- 2023Recommender Systems with Generative Retrievalcited by 198
- 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 198
- 2023Adapting Large Language Models by Integrating Collaborative Semantics for Recommendationcited by 179
- 2022Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning Systemcited by 112
- 2024Learnable Item Tokenization for Generative Recommendationcited by 110
- 2024Data-efficient Fine-tuning for LLM-based Recommendationcited by 104
- 2023LLM-Rec: Personalized Recommendation via Prompting Large Language Modelscited by 77
37 more
- 2024Discrete-state Continuous-time Diffusion for Graph Generationcited by 76
- 2023A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systemscited by 73
- 2024On Softmax Direct Preference Optimization for Recommendationcited by 64
- 2025SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learningcited by 63
- 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 56
- 2025Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMscited by 55
- 2024IDGenRec: LLM-RecSys Alignment with Textual ID Learningcited by 53
- 2024How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmarkcited by 43
- 2020GraphSAIL: Graph Structure Aware Incremental Learning for Recommender Systemscited by 42
- 2025Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrativecited by 40
- 2023Continual Learning on Dynamic Graphs via Parameter Isolationcited by 36
- 2020One Person, One Model, One World: Learning Continual User Representation without Forgettingcited by 30
- 2023Class-Imbalanced Graph Learning without Class Rebalancingcited by 29
- 2023Ensuring User-side Fairness in Dynamic Recommender Systemscited by 27
- 2023Dynamically Expandable Graph Convolution for Streaming Recommendationcited by 26
- 2020ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendationcited by 22
- 2025Order-agnostic Identifier for Large Language Model-based Generative Recommendationcited by 21
- 2025Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuningcited by 21
- 2024Aligning Large Language Models with Recommendation Knowledgecited by 19
- 2024BACKTIME: Backdoor Attacks on Multivariate Time Series Forecastingcited by 18
- 2025SelfElicit: Your Language Model Secretly Knows Where is the Relevant Evidencecited by 18
- 2025Pave Your Own Path: Graph Gradual Domain Adaptation on Fused Gromov-Wasserstein Geodesicscited by 17
- 2025TUCKET: A Tensor Time Series Data Structure for Efficient and Accurate Factor Analysis over Time Rangescited by 12
- 2025Uncertainty Quantification and Decomposition for LLM-based Recommendationcited by 12
- 2025Breaking Silos: Adaptive Model Fusion Unlocks Better Time Series Forecastingcited by 12
- 2024Learning Attentional Mixture of LoRAs for Language Model Continual Learningcited by 10
- 2025Hierarchical LoRA MoE for Efficient CTR Model Scalingcited by 10
- 2023Structure Aware Incremental Learning with Personalized Imitation Weights for Recommender Systemscited by 9
- 2025Latte: Collaborative Test-Time Adaptation of Vision-Language Models in Federated Learningcited by 9
- 2023Preliminary Study on Incremental Learning for Large Language Model-based Recommender Systemscited by 7
- 2024CoRA: Collaborative Information Perception by Large Language Model's Weights for Recommendationcited by 7
- 2025Graph Data Selection for Domain Adaptation: A Model-Free Approachcited by 7
- 2023ReLoop2: Building Self-Adaptive Recommendation Models via Responsive Error Compensation Loopcited by 5
- 2022Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systemscited by 3
- 2025Capturing User Interests from Data Streams for Continual Sequential Recommendationcited by 3
- 2025Continual Recommender Systemscited by 2
- 2025SPRINT: Scalable and Predictive Intent Refinement for LLM-Enhanced Session-based Recommendationcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.