mlsanityData to 7 Oct 2026

Steering Prototypes with Prompt-tuning for Rehearsal-free Continual Learning

arXiv 2303.09447 ↗First posted 16 Mar 2023cs.LG cs.AI cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

In the context of continual learning, prototypes-as representative class embeddings-offer advantages in memory conservation and the mitigation of catastrophic forgetting. However, challenges related to semantic drift and prototype interference persist. In this study, we introduce the Contrastive Prototypical Prompt (CPP) approach. Through task-specific prompt-tuning, underpinned by a contrastive learning objective, we effectively address both aforementioned challenges. Our evaluations on four challenging class-incremental benchmarks reveal that CPP achieves a significant 4% to 6% improvement over state-of-the-art methods. Importantly, CPP operates without a rehearsal buffer and narrows the performance divergence between continual and offline joint-learning, suggesting an innovative scheme for Transformer-based continual learning systems.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 48 of its 75 references that are in the map. First posted on arXiv 16 Mar 2023.

Builds on

48 of 75 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  10. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  11. 2020Supervised Contrastive Learningcited by 2,803
  12. 2016Learning without Forgettingcited by 2,465
36 more
  1. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  2. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  3. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  4. 2022Visual Prompt Tuningcited by 1,674
  5. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  6. 2016Progressive Neural Networkscited by 1,332
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  8. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  9. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  10. 2021Learning to Prompt for Continual Learningcited by 938
  11. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  12. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  13. 2019Large Scale Incremental Learningcited by 727
  14. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  15. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  16. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  17. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  18. 2019Orthogonal Gradient Descent for Continual Learningcited by 361
  19. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  20. 2021Gradient Projection Memory for Continual Learningcited by 338
  21. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  22. 2019On Tiny Episodic Memories in Continual Learningcited by 299
  23. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  24. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  25. 2021Co2L: Contrastive Continual Learningcited by 243
  26. 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
  27. 2020Supermasks in Superpositioncited by 223
  28. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 184
  29. 2020Continual Prototype Evolution: Learning Online from Non-Stationary Data Streamscited by 143
  30. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 125
  31. 2019Functional Regularisation for Continual Learning with Gaussian Processescited by 121
  32. 2020Continual Deep Learning by Functional Regularisation of Memorable Pastcited by 104
  33. 2018Measuring and regularizing networks in function spacecited by 97
  34. 2022Probing Representation Forgetting in Supervised and Unsupervised Continual Learningcited by 74
  35. 2022Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interferencecited by 44
  36. 2020Energy-Based Models for Continual Learningcited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 6; 2025: 7; 2026: 2.

22 papers.

Sort:

By area:

  1. 2023Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimalitycited by 121
  2. 2023Continual Learning: Applications and the Road Forwardcited by 59
  3. 2023Prompt Engineering for Healthcare: Methodologies and Applicationscited by 52
  4. 2023A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithmcited by 32
  5. 2024Visual Prompt Tuning in Null Space for Continual Learningcited by 30
  6. 2025Language Guided Concept Bottleneck Models for Interpretable Continual Learningcited by 28
  7. 2023DeFormer: Integrating Transformers with Deformable Models for 3D Shape Abstraction from a Single Imagecited by 11
  8. 2025ProtoDepth: Unsupervised Continual Depth Completion with Prototypescited by 11
  9. 2024Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applicationscited by 7
  10. 2025Continual Learning on CLIP via Incremental Prompt Tuning with Intrinsic Textual Anchorscited by 6
  11. 2023Deep Deformable Models: Learning 3D Shape Abstractions with Part Consistencycited by 5
  12. 2023Federated Class-Incremental Learning with Promptingcited by 5
10 more
  1. 2024Instantaneous Perception of Moving Objects in 3Dcited by 3
  2. 2024Leveraging Hierarchical Taxonomies in Prompt-based Continual Learningcited by 3
  3. 2024Masked Autoencoders are Parameter-Efficient Federated Continual Learnerscited by 3
  4. 2025MGPATH: Vision-Language Model with Multi-Granular Prompt Learning for Few-Shot WSI Classificationcited by 3
  5. 2024Incremental Online Learning of Randomized Neural Network with Forward Regularizationcited by 2
  6. 2025Augmented Contrastive Clustering with Uncertainty-Aware Prototyping for Time Series Test Time Adaptationcited by 1
  7. 2025VIPAMIN: Visual Prompt Initialization via Embedding Selection and Subspace Expansioncited by 1
  8. 2025Large Sign Language Models: Toward 3D American Sign Language Translationcited by 1
  9. 2026Scalable Analytic Classifiers with Associative Drift Compensation for Class-Incremental Learning of Vision Transformerscited by 0
  10. 2026Revisiting Prototype Rehearsal for Exemplar-Free Continual Learning: Manifold-Aware Boundary Sampling with Adaptive Class-Balanced Losscited by 0