# PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learning

arXiv: https://arxiv.org/abs/2401.02094 · HTML: https://mlsanity.com/paper/2401.02094/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 81 references that are in the map.
- First posted on arXiv 4 Jan 2024.

## Builds on (most cited first, 30 of 59)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [Towards a Unified View of Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2110.04366/) (2021)
- [DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning](https://mlsanity.com/paper/2204.04799/) (2022)
- [The Future of Digital Health with Federated Learning](https://mlsanity.com/paper/2003.08119/) (2020)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning](https://mlsanity.com/paper/2211.13218/) (2022)
- [DER: Dynamically Expandable Representation for Class Incremental Learning](https://mlsanity.com/paper/2103.16788/) (2021)
- [FedProto: Federated Prototype Learning across Heterogeneous Clients](https://mlsanity.com/paper/2105.00243/) (2021)
- [Orthogonal Subspace Learning for Language Model Continual Learning](https://mlsanity.com/paper/2310.14152/) (2023)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)
- [No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data](https://mlsanity.com/paper/2106.05001/) (2021)

## Built on by (most cited first, 19 of 19)

- [CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning](https://mlsanity.com/paper/2505.24816/) (2025)
- [Closed-form merging of parameter-efficient modules for Federated Continual Learning](https://mlsanity.com/paper/2410.17961/) (2024)
- [Federated Continual Instruction Tuning](https://mlsanity.com/paper/2503.12897/) (2025)
- [Federated Class-Incremental Learning with Hierarchical Generative Prototypes](https://mlsanity.com/paper/2406.02447/) (2024)
- [FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2510.20250/) (2025)
- [Masked Autoencoders are Parameter-Efficient Federated Continual Learners](https://mlsanity.com/paper/2411.01916/) (2024)
- [Global Convergence of Continual Learning on Non-IID Data](https://mlsanity.com/paper/2503.18511/) (2025)
- [C^2Prompt: Class-aware Client Knowledge Interaction for Federated Continual Learning](https://mlsanity.com/paper/2509.19674/) (2025)
- [Task-agnostic Low-rank Residual Adaptation for Efficient Federated Continual Fine-Tuning](https://mlsanity.com/paper/2505.12318/) (2025)
- [Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection](https://mlsanity.com/paper/2509.21606/) (2025)
- [REAL-OW: Rehearsal-free Open World Object Detection with Low-Rank Adaptation and Dual-Stage Objectness Modeling](https://mlsanity.com/paper/2607.03004/) (2026)
- [Large-Small Model Collaborative Framework for Federated Continual Learning](https://mlsanity.com/paper/2508.09489/) (2025)
- [Deep Generative Continual Learning using Functional LoRA: FunLoRA](https://mlsanity.com/paper/2510.02631/) (2025)
- [DOLFIN: Balancing Stability and Plasticity in Federated Continual Learning](https://mlsanity.com/paper/2510.13567/) (2025)
- [MLFCIL: A Multi-Level Forgetting Mitigation Framework for Federated Class-Incremental Learning in LEO Satellites](https://mlsanity.com/paper/2604.02356/) (2026)
- [Fisher-Routed Mixture of Experts for Federated Class-Incremental Learning](https://mlsanity.com/paper/2606.28835/) (2026)
- [HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learning](https://mlsanity.com/paper/2607.08784/) (2026)
- [SUM: Unified Geometric Surgery on Spatio-Temporal Adaptation Vectors for Federated Class Incremental Learning](https://mlsanity.com/paper/2607.19384/) (2026)
- [ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learning](https://mlsanity.com/paper/2609.38833/) (2026)

## Abstract (arXiv)

Existing federated learning methods have effectively dealt with decentralized learning in scenarios involving data privacy and non-IID data. However, in real-world situations, each client dynamically learns new classes, requiring the global model to classify all seen classes. To effectively mitigate catastrophic forgetting and data heterogeneity under low communication costs, we propose a simple and effective method named PILoRA. On the one hand, we adopt prototype learning to learn better feature representations and leverage the heuristic information between prototypes and class features to design a prototype re-weight module to solve the classifier bias caused by data heterogeneity without retraining the classifier. On the other hand, we view incremental learning as the process of learning distinct task vectors and encoding them within different LoRA parameters. Accordingly, we propose Incremental LoRA to mitigate catastrophic forgetting. Experimental results on standard datasets indicate that our method outperforms the state-of-the-art approaches significantly. More importantly, our method exhibits strong robustness and superiority in different settings and degrees of data heterogeneity. The code is available at urlhttps://github.com/Ghy0501/PILoRA.