# CAPrompt: Cyclic Prompt Aggregation for Pre-Trained Model Based Class Incremental Learning

arXiv: https://arxiv.org/abs/2412.08929 · HTML: https://mlsanity.com/paper/2412.08929/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters](https://mlsanity.com/paper/2403.11549/) (2024)
- [InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning](https://mlsanity.com/paper/2404.00228/) (2024)
- [Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning](https://mlsanity.com/paper/2106.09701/) (2021)
- [Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality](https://mlsanity.com/paper/2310.07234/) (2023)
- [RMM: Reinforced Memory Management for Class-Incremental Learning](https://mlsanity.com/paper/2301.05792/) (2023)
- [Consistent Prompting for Rehearsal-Free Continual Learning](https://mlsanity.com/paper/2403.08568/) (2024)
- [Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interference](https://mlsanity.com/paper/2211.15969/) (2022)
- [Convolutional Prompting meets Language Models for Continual Learning](https://mlsanity.com/paper/2403.20317/) (2024)
- [Continual Adapter Tuning with Semantic Shift Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2403.19979/) (2024)

## Built on by (most cited first, 9 of 9)

- [GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model](https://mlsanity.com/paper/2505.04119/) (2025)
- [EKPC: Elastic Knowledge Preservation and Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2506.12351/) (2025)
- [Token Coordinated Prompt Attention is Needed for Visual Prompting](https://mlsanity.com/paper/2505.02406/) (2025)
- [Vision Graph Prompting via Semantic Low-Rank Decomposition](https://mlsanity.com/paper/2505.04121/) (2025)
- [HERMAN: Hierarchical Representation Matching for CLIP-based Class-Incremental Learning](https://mlsanity.com/paper/2509.22645/) (2025)
- [The Lie of the Average: How Class Incremental Learning Evaluation Deceives You?](https://mlsanity.com/paper/2509.22580/) (2025)
- [Unlocking Positive Transfer in Incrementally Learning Surgical Instruments: A Self-reflection Hierarchical Prompt Framework](https://mlsanity.com/paper/2604.02877/) (2026)
- [Teaching Prompts to Coordinate: Hierarchical Layer-Grouped Prompt Tuning for Continual Learning](https://mlsanity.com/paper/2511.12090/) (2025)
- [Unlocking Prototype Potential: An Efficient Tuning Framework for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2602.05271/) (2026)

## Abstract (arXiv)

Recently, prompt tuning methods for pre-trained models have demonstrated promising performance in Class Incremental Learning (CIL). These methods typically involve learning task-specific prompts and predicting the task ID to select the appropriate prompts for inference. However, inaccurate task ID predictions can cause severe inconsistencies between the prompts used during training and inference, leading to knowledge forgetting and performance degradation. Additionally, existing prompt tuning methods rely solely on the pre-trained model to predict task IDs, without fully leveraging the knowledge embedded in the learned prompt parameters, resulting in inferior prediction performance. To address these issues, we propose a novel Cyclic Prompt Aggregation (CAPrompt) method that eliminates the dependency on task ID prediction by cyclically aggregating the knowledge from different prompts. Specifically, rather than predicting task IDs, we introduce an innovative prompt aggregation strategy during both training and inference to overcome prompt inconsistency by utilizing a weighted sum of different prompts. Thorough theoretical analysis demonstrates that under concave conditions, the aggregated prompt achieves lower error compared to selecting a single task-specific prompt. Consequently, we incorporate a concave constraint and a linear constraint to guide prompt learning, ensuring compliance with the concave condition requirement. Furthermore, to fully exploit the prompts and achieve more accurate prompt weights, we develop a cyclic weight prediction strategy. This strategy begins with equal weights for each task and automatically adjusts them to more appropriate values in a cyclical manner. Experiments on various datasets demonstrate that our proposed CAPrompt outperforms state-of-the-art methods by 2%-3%. Our code is available at https://github.com/zhoujiahuan1991/AAAI2025-CAPrompt.