mlsanityData to 7 Oct 2026

Boosting Open-Domain Continual Learning via Leveraging Intra-domain Category-aware Prototype

arXiv 2408.09984 ↗First posted 19 Aug 2024cs.CVcited by 4

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Despite recent progress in enhancing the efficacy of Open-Domain Continual Learning (ODCL) in Vision-Language Models (VLM), failing to (1) correctly identify the Task-ID of a test image and (2) use only the category set corresponding to the Task-ID, while preserving the knowledge related to each domain, cannot address the two primary challenges of ODCL: forgetting old knowledge and maintaining zero-shot capabilities, as well as the confusions caused by category-relatedness between domains. In this paper, we propose a simple yet effective solution: leveraging intra-domain category-aware prototypes for ODCL in CLIP (DPeCLIP), where the prototype is the key to bridging the above two processes. Concretely, we propose a training-free Task-ID discriminator method, by utilizing prototypes as classifiers for identifying Task-IDs. Furthermore, to maintain the knowledge corresponding to each domain, we incorporate intra-domain category-aware prototypes as domain prior prompts into the training process. Extensive experiments conducted on 11 different datasets demonstrate the effectiveness of our approach, achieving 2.37% and 1.14% average improvement in class-incremental and task-incremental settings, respectively.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 40 references that are in the map. First posted on arXiv 19 Aug 2024.

Builds on

20 of 40 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters18 Mar
  2. 2024CoLeCLIP: Open-Domain Continual Learning via Joint Task Prompt and Vocabulary Learning15 Mar

Other references · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2016Learning without Forgettingcited by 2,465
  8. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  9. 2013Describing Textures in the Wildcited by 2,110
  10. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  12. 2017Continual Learning with Deep Generative Replaycited by 1,152
6 more
  1. 2021Learning to Prompt for Continual Learningcited by 938
  2. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  3. 2018Learning without Memorizingcited by 249
  4. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  5. 2023A Unified Continual Learning Framework with General Parameter-Efficient Tuningcited by 124
  6. 2022Don't Stop Learning: Towards Continual Learning for the CLIP Modelcited by 61

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024Recent Advances of Multimodal Continual Learning: A Comprehensive Surveycited by 39
  2. 2025Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgettingcited by 16
  3. 2025Enhanced Continual Learning of Vision-Language Models with Model Fusioncited by 3
  4. 2026Enhancing Continual Learning of Vision-Language Models via Dynamic Prefix Weightingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models
  2. 2025Preserving Cross-Modal Consistency for CLIP-based Class-Incremental Learning
  3. 2023CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservation
  4. 2025Harnessing Textual Semantic Priors for Knowledge Transfer and Refinement in CLIP-Driven Continual Learning
  5. 2025Beyond CLIP Generalization: Against Forward&Backward Forgetting Adapter for Continual Learning of Vision-Language Models
  6. 2026Black-Box Continual Learning for Vision-Language Models
  7. 2025ChordPrompt: Orchestrating Cross-Modal Prompt Synergy for Multi-Domain Incremental Learning in CLIP
  8. 2024Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Models