PIP: Prototypes-Injected Prompt for Federated Class Incremental Learning
Not on the map. Why?
Abstract (arXiv)
Federated Class Incremental Learning (FCIL) is a new direction in continual learning (CL) for addressing catastrophic forgetting and non-IID data distribution simultaneously. Existing FCIL methods call for high communication costs and exemplars from previous classes. We propose a novel rehearsal-free method for FCIL named prototypes-injected prompt (PIP) that involves 3 main ideas: a) prototype injection on prompt learning, b) prototype augmentation, and c) weighted Gaussian aggregation on the server side. Our experiment result shows that the proposed method outperforms the current state of the arts (SOTAs) with a significant improvement (up to 33%) in CIFAR100, MiniImageNet and TinyImageNet datasets. Our extensive analysis demonstrates the robustness of PIP in different task sizes, and the advantage of requiring smaller participating local clients, and smaller global rounds. For further study, source codes of PIP, baseline, and experimental logs are shared publicly in https://github.com/anwarmaxsum/PIP.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 30 Jul 2024.
Builds on
31 of 42 references.
References · 31
- 2017Attention Is All You Needcited by 59,932
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2016Learning without Forgettingcited by 2,465
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
- 2017Continual Learning with Deep Generative Replaycited by 1,152
- 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
- 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
19 more
- 2021Learning to Prompt for Continual Learningcited by 938
- 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
- 2019Large Scale Incremental Learningcited by 727
- 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
- 2018End-to-End Incremental Learningcited by 596
- 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
- 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
- 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
- 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
- 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
- 2019Overcoming Forgetting in Federated Learning on Non-IID Datacited by 120
- 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
- 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidationcited by 113
- 2021On Learning the Geodesic Path for Incremental Learningcited by 85
- 2023Better Generative Replay for Continual Federated Learningcited by 53
- 2021A distillation-based approach integrating continual learning and federated learning for pervasive servicescited by 42
- 2022Towards the Practical Utility of Federated Learning in the Medical Domaincited by 9
- 2023Assessor-Guided Learning for Continual Environmentscited by 7
- 2023Federated Class-Incremental Learning with Promptingcited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.