mlsanityData to 7 Oct 2026

DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning

arXiv 2204.04799 ↗First posted 10 Apr 2022cs.LG cs.CVcited by 635

Not on the map. Why?

Cited by 65 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Continual learning aims to enable a single model to learn a sequence of tasks without catastrophic forgetting. Top-performing methods usually require a rehearsal buffer to store past pristine examples for experience replay, which, however, limits their practical value due to privacy and memory constraints. In this work, we present a simple yet effective framework, DualPrompt, which learns a tiny set of parameters, called prompts, to properly instruct a pre-trained model to learn tasks arriving sequentially without buffering past examples. DualPrompt presents a novel approach to attach complementary prompts to the pre-trained backbone, and then formulates the objective as learning task-invariant and task-specific "instructions". With extensive experimental validation, DualPrompt consistently sets state-of-the-art performance under the challenging class-incremental setting. In particular, DualPrompt outperforms recent advanced continual learning methods with relatively large buffer sizes. We also introduce a more challenging benchmark, Split ImageNet-R, to help generalize rehearsal-free continual learning research. Source code is available at https://github.com/google-research/l2p.

Key facts. Cited by 635 papers in the map (data through 7 Oct 2026). Builds on 52 of its 72 references that are in the map. First posted on arXiv 10 Apr 2022.

Builds on

52 of 72 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  8. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  11. 2016Learning without Forgettingcited by 2,465
  12. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
38 more
  1. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  2. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  3. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  4. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  5. 2016Progressive Neural Networkscited by 1,332
  6. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  7. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  8. 2021Learning to Prompt for Continual Learningcited by 938
  9. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  10. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  11. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  12. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  13. 2019Large Scale Incremental Learningcited by 727
  14. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  15. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  16. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  17. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  18. 2021ImageNet-21K Pretraining for the Massescited by 607
  19. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 446
  20. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  21. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 318
  22. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  23. 2021Co2L: Contrastive Continual Learningcited by 243
  24. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  25. 2020Supermasks in Superpositioncited by 223
  26. 2020Using Hindsight to Anchor Past Knowledge in Continual Learningcited by 160
  27. 2020Adversarial Continual Learningcited by 140
  28. 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
  29. 2018Memory Efficient Experience Replay for Streaming Learningcited by 128
  30. 2022Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning Systemcited by 110
  31. 2021Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understandingcited by 97
  32. 2021Continual Learning of a Mixed Sequence of Similar and Dissimilar Taskscited by 80
  33. 2020Generalized Variational Continual Learningcited by 56
  34. 2019Rehearsal-Free Continual Learning over Small Non-I.I.D. Batchescited by 46
  35. 2022Architecture Matters in Continual Learningcited by 46
  36. 2021DualNet: Continual Learning, Fast and Slowcited by 39
  37. 2020Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updatescited by 34
  38. 2020Learn-Prune-Share for Lifelong Learningcited by 32

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 21; 2023: 101; 2024: 176; 2025: 191; 2026: 146.

635 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  3. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  4. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 267
  5. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  6. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  7. 2023Class-Incremental Learning: A Surveycited by 209
  8. 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 189
  9. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  10. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  11. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
  12. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 170
38 more
  1. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 162
  2. 2024Continual Learning with Pre-Trained Models: A Surveycited by 151
  3. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 138
  4. 2023A Unified Continual Learning Framework with General Parameter-Efficient Tuningcited by 124
  5. 2023Multimodal Prompting with Missing Modalities for Visual Recognitioncited by 121
  6. 2023Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimalitycited by 121
  7. 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 115
  8. 2023PromptIR: Prompting for All-in-One Blind Image Restorationcited by 113
  9. 2023Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuningcited by 106
  10. 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 92
  11. 2023FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learningcited by 86
  12. 2023Learning without Forgetting for Vision-Language Modelscited by 79
  13. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  14. 2022CLIP model is an Efficient Continual Learnercited by 64
  15. 2023PILOT: A Pre-Trained Model-Based Continual Learning Toolboxcited by 64
  16. 2025SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learningcited by 60
  17. 2023Read-only Prompt Optimization for Vision-Language Few-shot Learningcited by 58
  18. 2022SparCL: Sparse Continual Learning on the Edgecited by 55
  19. 2022A Simple Baseline that Questions the Use of Pretrained-Models in Continual Learningcited by 54
  20. 2024Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learningcited by 53
  21. 2024Consistent Prompting for Rehearsal-Free Continual Learningcited by 52
  22. 2023Introducing Language Guidance in Prompt-based Continual Learningcited by 49
  23. 2023Continual Instruction Tuning for Large Multimodal Modelscited by 49
  24. 2023A Survey on Few-Shot Class-Incremental Learningcited by 48
  25. 2024Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusioncited by 47
  26. 2023First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learningcited by 46
  27. 2023AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learningcited by 46
  28. 2024Recent Advances of Foundation Language Models-based Continual Learning: A Surveycited by 46
  29. 2022PIVOT: Prompting for Video Continual Learningcited by 45
  30. 2023Dense Network Expansion for Class Incremental Learningcited by 45
  31. 2022Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interferencecited by 44
  32. 2024Mixture of Experts Meets Prompt-Based Continual Learningcited by 43
  33. 2023CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Updatecited by 42
  34. 2023Weighted Ensemble Models Are Strong Continual Learnerscited by 41
  35. 2024Recent Advances of Multimodal Continual Learning: A Comprehensive Surveycited by 39
  36. 2023TiC-CLIP: Continual Training of CLIP Modelscited by 38
  37. 2023Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuningcited by 35
  38. 2024Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learnerscited by 35

50 most cited of 635 All 635 (JSON)