Is Parameter Isolation Better for Prompt-Based Continual Learning?
Not on the map. Why?
Abstract (arXiv)
Prompt-based continual learning methods effectively mitigate catastrophic forgetting. However, most existing methods assign a fixed set of prompts to each task, completely isolating knowledge across tasks and resulting in suboptimal parameter utilization. To address this, we consider the practical needs of continual learning and propose a prompt-sharing framework. This framework constructs a global prompt pool and introduces a task-aware gated routing mechanism that sparsely activates a subset of prompts to achieve dynamic decoupling and collaborative optimization of task-specific feature representations. Furthermore, we introduce a history-aware modulator that leverages cumulative prompt activation statistics to protect frequently used prompts from excessive updates, thereby mitigating inefficient parameter usage and knowledge forgetting. Extensive analysis and empirical results demonstrate that our approach consistently outperforms existing static allocation strategies in effectiveness and efficiency.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 52 references that are in the map. First posted on arXiv 28 Jan 2026.
Builds on
35 of 52 references.
References · 35
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2016Learning without Forgettingcited by 2,496
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
- 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
- 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
- 2021Learning to Prompt for Continual Learningcited by 944
- 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
- 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
- 2021ImageNet-21K Pretraining for the Massescited by 613
23 more
- 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 450
- 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
- 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
- 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
- 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 198
- 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 184
- 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 180
- 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 164
- 2024Continual Learning with Pre-Trained Models: A Surveycited by 154
- 2020Adversarial Continual Learningcited by 142
- 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
- 2023A Unified Continual Learning Framework with General Parameter-Efficient Tuningcited by 125
- 2023Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimalitycited by 121
- 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 117
- 2020A Comprehensive Study of Class Incremental Learning Algorithms for Visual Taskscited by 107
- 2022Memory Replay with Data Compression for Continual Learningcited by 82
- 2024Consistent Prompting for Rehearsal-Free Continual Learningcited by 54
- 2023First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learningcited by 46
- 2024Mixture of Experts Meets Prompt-Based Continual Learningcited by 43
- 2024Convolutional Prompting meets Language Models for Continual Learningcited by 32
- 2024Beyond Prompt Learning: Continual Adapter for Efficient Rehearsal-Free Continual Learningcited by 29
- 2024OVOR: OnePrompt with Virtual Outlier Regularization for Rehearsal-Free Class-Incremental Learningcited by 20
- 2023Cross-Class Feature Augmentation for Class Incremental Learningcited by 8
Built on by
Citing papers per year: 2026: 1.
1 paper.