mlsanityData to 14 Oct 2026

Is Parameter Isolation Better for Prompt-Based Continual Learning?

arXiv 2601.20894 ↗First posted 28 Jan 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Prompt-based continual learning methods effectively mitigate catastrophic forgetting. However, most existing methods assign a fixed set of prompts to each task, completely isolating knowledge across tasks and resulting in suboptimal parameter utilization. To address this, we consider the practical needs of continual learning and propose a prompt-sharing framework. This framework constructs a global prompt pool and introduces a task-aware gated routing mechanism that sparsely activates a subset of prompts to achieve dynamic decoupling and collaborative optimization of task-specific feature representations. Furthermore, we introduce a history-aware modulator that leverages cumulative prompt activation statistics to protect frequently used prompts from excessive updates, thereby mitigating inefficient parameter usage and knowledge forgetting. Extensive analysis and empirical results demonstrate that our approach consistently outperforms existing static allocation strategies in effectiveness and efficiency.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 52 references that are in the map. First posted on arXiv 28 Jan 2026.

Builds on

35 of 52 references.

References · 35

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  5. 2016Learning without Forgettingcited by 2,496
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  8. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  9. 2021Learning to Prompt for Continual Learningcited by 944
  10. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  11. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
  12. 2021ImageNet-21K Pretraining for the Massescited by 613
23 more
  1. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 450
  2. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
  3. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  4. 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
  5. 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 198
  6. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 184
  7. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 180
  8. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 164
  9. 2024Continual Learning with Pre-Trained Models: A Surveycited by 154
  10. 2020Adversarial Continual Learningcited by 142
  11. 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
  12. 2023A Unified Continual Learning Framework with General Parameter-Efficient Tuningcited by 125
  13. 2023Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimalitycited by 121
  14. 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 117
  15. 2020A Comprehensive Study of Class Incremental Learning Algorithms for Visual Taskscited by 107
  16. 2022Memory Replay with Data Compression for Continual Learningcited by 82
  17. 2024Consistent Prompting for Rehearsal-Free Continual Learningcited by 54
  18. 2023First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learningcited by 46
  19. 2024Mixture of Experts Meets Prompt-Based Continual Learningcited by 43
  20. 2024Convolutional Prompting meets Language Models for Continual Learningcited by 32
  21. 2024Beyond Prompt Learning: Continual Adapter for Efficient Rehearsal-Free Continual Learningcited by 29
  22. 2024OVOR: OnePrompt with Virtual Outlier Regularization for Rehearsal-Free Class-Incremental Learningcited by 20
  23. 2023Cross-Class Feature Augmentation for Class Incremental Learningcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.