# CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning

arXiv: https://arxiv.org/abs/2505.24816 · HTML: https://mlsanity.com/paper/2505.24816/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 63 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 30 of 50)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning](https://mlsanity.com/paper/2204.04799/) (2022)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning](https://mlsanity.com/paper/2211.13218/) (2022)
- [A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark](https://mlsanity.com/paper/1910.04867/) (2019)
- [AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2303.10512/) (2023)
- [Orthogonal Subspace Learning for Language Model Continual Learning](https://mlsanity.com/paper/2310.14152/) (2023)
- [How Do Vision Transformers Work?](https://mlsanity.com/paper/2202.06709/) (2022)
- [Maintaining Discrimination and Fairness in Class Incremental Learning](https://mlsanity.com/paper/1911.07053/) (2019)
- [DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation](https://mlsanity.com/paper/2210.07558/) (2022)
- [FOSTER: Feature Boosting and Compression for Class-Incremental Learning](https://mlsanity.com/paper/2204.04662/) (2022)
- [Mnemonics Training: Multi-Class Incremental Learning without Forgetting](https://mlsanity.com/paper/2002.10211/) (2020)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning](https://mlsanity.com/paper/2404.00228/) (2024)
- [Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need](https://mlsanity.com/paper/2303.07338/) (2023)
- [RanPAC: Random Projections and Pre-trained Models for Continual Learning](https://mlsanity.com/paper/2307.02251/) (2023)
- [Continual Learning with Pre-Trained Models: A Survey](https://mlsanity.com/paper/2401.16386/) (2024)

## Built on by (most cited first, 28 of 28)

- [Lifecycle-Aware Federated Continual Learning in Mobile Autonomous Systems](https://mlsanity.com/paper/2604.20745/) (2026)
- [CKAA: Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learning](https://mlsanity.com/paper/2507.09471/) (2025)
- [Revisiting Weight Regularization for Low-Rank Continual Learning](https://mlsanity.com/paper/2602.17559/) (2026)
- [Continual Knowledge Consolidation LORA for Domain Incremental Learning](https://mlsanity.com/paper/2510.16077/) (2025)
- [C-Flat++: Towards a More Efficient and Powerful Framework for Continual Learning](https://mlsanity.com/paper/2508.18860/) (2025)
- [EW-DETR: Evolving World Object Detection via Incremental Low-Rank DEtection TRansformer](https://mlsanity.com/paper/2602.20985/) (2026)
- [One Adapter for All: Towards Unified Representation in Step-Imbalanced Class-Incremental Learning](https://mlsanity.com/paper/2603.10237/) (2026)
- [Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning](https://mlsanity.com/paper/2603.12816/) (2026)
- [CATFormer: When Continual Learning Meets Spiking Transformers With Dynamic Thresholds](https://mlsanity.com/paper/2603.15184/) (2026)
- [Live Interactive Training for Video Segmentation](https://mlsanity.com/paper/2603.26929/) (2026)
- [Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting](https://mlsanity.com/paper/2605.29498/) (2026)
- [Normality-Preserving Continual Industrial Anomaly Detection via Orthogonal LoRA Banks](https://mlsanity.com/paper/2606.02042/) (2026)
- [REAL-OW: Rehearsal-free Open World Object Detection with Low-Rank Adaptation and Dual-Stage Objectness Modeling](https://mlsanity.com/paper/2607.03004/) (2026)
- [Bilinear Optimization Divergence: Diagnosing Factor-Constrained LoRA Continual Learning](https://mlsanity.com/paper/2609.23594/) (2026)
- [Scalable Analytic Classifiers with Associative Drift Compensation for Class-Incremental Learning of Vision Transformers](https://mlsanity.com/paper/2602.00144/) (2026)
- [Advancing Analytic Class-Incremental Learning through Vision-Language Calibration](https://mlsanity.com/paper/2602.13670/) (2026)
- [Dual-Imbalance Continual Learning for Real-World Food Recognition](https://mlsanity.com/paper/2603.29133/) (2026)
- [Free-Flow Class-Incremental Learning: Towards Robust CIL under Variable Class Arrivals](https://mlsanity.com/paper/2604.02765/) (2026)
- [SR2-LoRA: Self-Rectifying Inter-layer Relations in Low-Rank Adaptation for Class-Incremental Learning](https://mlsanity.com/paper/2605.07420/) (2026)
- [iGSP:Implicit Gradient Subspace Projection for Efficient Continual Learning of Vision-Language Models](https://mlsanity.com/paper/2605.19301/) (2026)
- [Fisher-Routed Mixture of Experts for Federated Class-Incremental Learning](https://mlsanity.com/paper/2606.28835/) (2026)
- [Few-Shot Domain Incremental Learning via Continual Vision-Language Consolidation](https://mlsanity.com/paper/2606.30190/) (2026)
- [C2Path: Class-Conditional Pathway Decoupling for Vision-Language Incremental Object Detection](https://mlsanity.com/paper/2608.21937/) (2026)
- [Adapter-Based Few-Shot Continual Learning for Malicious Packet Recognition](https://mlsanity.com/paper/2608.23536/) (2026)
- [Parameter Efficient Continual Learning for Sparse Event-Based Transformers](https://mlsanity.com/paper/2608.26720/) (2026)
- [When the Merge Coefficient Stops Mattering: Proximity Regularized Merging for Continual LoRA Adaptation](https://mlsanity.com/paper/2609.32332/) (2026)
- [Two Routes to the Middle: Placement Search and Brain Readouts Converge on Where Continual Learners Should Specialize](https://mlsanity.com/paper/2610.01590/) (2026)
- [Stable-Plastic Subspace Construction for Differentially Private Continual LoRA](https://mlsanity.com/paper/2610.17904/) (2026)

## Abstract (arXiv)

Class-Incremental Learning (CIL) aims to learn new classes sequentially while retaining the knowledge of previously learned classes. Recently, pre-trained models (PTMs) combined with parameter-efficient fine-tuning (PEFT) have shown remarkable performance in rehearsal-free CIL without requiring exemplars from previous tasks. However, existing adapter-based methods, which incorporate lightweight learnable modules into PTMs for CIL, create new adapters for each new task, leading to both parameter redundancy and failure to leverage shared knowledge across tasks. In this work, we propose ContinuaL Low-Rank Adaptation (CL-LoRA), which introduces a novel dual-adapter architecture combining task-shared adapters to learn cross-task knowledge and task-specific adapters to capture unique features of each new task. Specifically, the shared adapters utilize random orthogonal matrices and leverage knowledge distillation with gradient reassignment to preserve essential shared knowledge. In addition, we introduce learnable block-wise weights for task-specific adapters, which mitigate inter-task interference while maintaining the model's plasticity. We demonstrate CL-LoRA consistently achieves promising performance under multiple benchmarks with reduced training and inference computation, establishing a more efficient and scalable paradigm for continual learning with pre-trained models.