# DeCLIP: Decoupled Prompting for Multi-Label Class-Incremental Learning with CLIP

arXiv: https://arxiv.org/abs/2509.23335 · HTML: https://mlsanity.com/paper/2509.23335/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Builds on (most cited first, 16 of 16)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters](https://mlsanity.com/paper/2403.11549/) (2024)
- [TResNet: High Performance GPU-Dedicated Architecture](https://mlsanity.com/paper/2003.13630/) (2020)
- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [Imbalanced Continual Learning with Partitioning Reservoir Sampling](https://mlsanity.com/paper/2009.03632/) (2020)
- [CLIP model is an Efficient Continual Learner](https://mlsanity.com/paper/2210.03114/) (2022)
- [Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusion](https://mlsanity.com/paper/2407.14143/) (2024)
- [DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2308.01890/) (2023)
- [Harnessing Textual Semantic Priors for Knowledge Transfer and Refinement in CLIP-Driven Continual Learning](https://mlsanity.com/paper/2508.01579/) (2025)
- [Cut out and Replay: A Simple yet Versatile Strategy for Multi-Label Online Continual Learning](https://mlsanity.com/paper/2505.19680/) (2025)

## Abstract (arXiv)

Multi-label class-incremental learning (MLCIL) continuously expands the label space while recognizing multiple co-occurring categories, making catastrophic forgetting a central challenge. Recent class-incremental learning methods have increasingly adopted CLIP as their backbone. However, we find that applying CLIP to MLCIL exhibits two critical issues: entanglement of class-specific cues in shared visual representations and high false-positive rates (FPR) under task-level partial labeling. We propose DeCLIP, a replay-free and parameter-efficient framework for CLIP-based MLCIL. DeCLIP uses Decoupled Prompting to learn class-specific positive and negative prompts in both visual and textual modalities, enabling class-conditioned vision-language matching and reducing representation entanglement. Only new-category prompts are optimized, previous prompts remain unchanged, preserving prior knowledge and mitigating catastrophic forgetting without replay. DeCLIP further incorporates Adaptive Similarity Tempering, an inference-time strategy that adapts similarity-tempering strength to the incremental configuration, suppressing false positives without specific tuning. Experiments on MS-COCO, PASCAL VOC, and the real-world NUS-WIDEseq benchmark demonstrate consistent improvements over prior methods with a few trainable parameters.