# Continual Low-Rank Adapters for LLM-based Generative Recommender Systems

arXiv: https://arxiv.org/abs/2510.25093 · HTML: https://mlsanity.com/paper/2510.25093/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 74 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Builds on (most cited first, 30 of 49)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [Orthogonal Subspace Learning for Language Model Continual Learning](https://mlsanity.com/paper/2310.14152/) (2023)
- [Recommender Systems with Generative Retrieval](https://mlsanity.com/paper/2305.05065/) (2023)
- [InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning](https://mlsanity.com/paper/2404.00228/) (2024)
- [Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation](https://mlsanity.com/paper/2311.09049/) (2023)
- [Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System](https://mlsanity.com/paper/2201.12604/) (2022)
- [Learnable Item Tokenization for Generative Recommendation](https://mlsanity.com/paper/2405.07314/) (2024)
- [Data-efficient Fine-tuning for LLM-based Recommendation](https://mlsanity.com/paper/2401.17197/) (2024)
- [LLM-Rec: Personalized Recommendation via Prompting Large Language Models](https://mlsanity.com/paper/2307.15780/) (2023)
- [Discrete-state Continuous-time Diffusion for Graph Generation](https://mlsanity.com/paper/2405.11416/) (2024)
- [A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systems](https://mlsanity.com/paper/2308.08434/) (2023)
- [On Softmax Direct Preference Optimization for Recommendation](https://mlsanity.com/paper/2406.09215/) (2024)
- [SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning](https://mlsanity.com/paper/2501.13198/) (2025)
- [Reliable Adversarial Distillation with Unreliable Teachers](https://mlsanity.com/paper/2106.04928/) (2021)
- [Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs](https://mlsanity.com/paper/2510.17771/) (2025)
- [IDGenRec: LLM-RecSys Alignment with Textual ID Learning](https://mlsanity.com/paper/2403.19021/) (2024)
- [How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark](https://mlsanity.com/paper/2406.06647/) (2024)
- [GraphSAIL: Graph Structure Aware Incremental Learning for Recommender Systems](https://mlsanity.com/paper/2008.13517/) (2020)
- [Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative](https://mlsanity.com/paper/2502.08942/) (2025)
- [Continual Learning on Dynamic Graphs via Parameter Isolation](https://mlsanity.com/paper/2305.13825/) (2023)
- [One Person, One Model, One World: Learning Continual User Representation without Forgetting](https://mlsanity.com/paper/2009.13724/) (2020)
- [Class-Imbalanced Graph Learning without Class Rebalancing](https://mlsanity.com/paper/2308.14181/) (2023)
- [Ensuring User-side Fairness in Dynamic Recommender Systems](https://mlsanity.com/paper/2308.15651/) (2023)
- [Dynamically Expandable Graph Convolution for Streaming Recommendation](https://mlsanity.com/paper/2303.11700/) (2023)
- [ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation](https://mlsanity.com/paper/2007.12000/) (2020)
- [Order-agnostic Identifier for Large Language Model-based Generative Recommendation](https://mlsanity.com/paper/2502.10833/) (2025)
- [Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning](https://mlsanity.com/paper/2505.16270/) (2025)

## Built on by (most cited first, 2 of 2)

- [Awakening Dormant Users: Generative Recommendation with Counterfactual Functional Role Reasoning](https://mlsanity.com/paper/2602.13134/) (2026)
- [Self-Evolving Memory for Generative Recommendation](https://mlsanity.com/paper/2609.15598/) (2026)

## Abstract (arXiv)

While large language models (LLMs) achieve strong performance in recommendation, they face challenges in continual learning as users, items, and user preferences evolve over time. Existing LoRA-based continual methods primarily focus on preserving performance on previous tasks, but this overlooks the unique nature of recommendation: the goal is not to predict past preferences, and outdated preferences can even harm performance when current interests shift significantly. To address this, we propose PESO (Proximally rEgularized Single evolving lOra, a continual adaptation method for LoRA in recommendation. PESO introduces a proximal regularizer that anchors the current adapter to its most recent frozen state, enabling the model to flexibly balance adaptation and preservation, and to better capture recent user behaviors. Theoretically, we show that this proximal design provides data-aware, direction-wise guidance in the LoRA subspace. Empirically, PESO consistently outperforms existing LoRA-based continual learning methods.