# Federated Class-Incremental Learning with New-Class Augmented Self-Distillation

arXiv: https://arxiv.org/abs/2401.00622 · HTML: https://mlsanity.com/paper/2401.00622/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Federated, robust and private distillation, one of 12 directions in Knowledge distillation.
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 1 Jan 2024.

## Placed in

- Knowledge distillation / [Federated, robust and private distillation](https://mlsanity.com/distill/federated-robust-and-private-distillation/)

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [FedML: A Research Library and Benchmark for Federated Machine Learning](https://mlsanity.com/paper/2007.13518/) (2020)
- [Maintaining Discrimination and Fairness in Class Incremental Learning](https://mlsanity.com/paper/1911.07053/) (2019)
- [Mnemonics Training: Multi-Class Incremental Learning without Forgetting](https://mlsanity.com/paper/2002.10211/) (2020)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Large scale distributed neural network training through online distillation](https://mlsanity.com/paper/1804.03235/) (2018)
- [Federated Class-Incremental Learning](https://mlsanity.com/paper/2203.11473/) (2022)
- [Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data](https://mlsanity.com/paper/2008.06180/) (2020)
- [Preservation of the Global Knowledge by Not-True Distillation in Federated Learning](https://mlsanity.com/paper/2106.03097/) (2021)
- [No One Left Behind: Real-World Federated Class-Incremental Learning](https://mlsanity.com/paper/2302.00903/) (2023)
- [Knowledge Distillation for Federated Learning: a Practical Guide](https://mlsanity.com/paper/2211.04742/) (2022)
- [Towards Federated Learning against Noisy Labels via Local Self-Regularization](https://mlsanity.com/paper/2208.12807/) (2022)
- [FedET: A Communication-Efficient Federated Class-Incremental Learning Framework Based on Enhanced Transformer](https://mlsanity.com/paper/2306.15347/) (2023)
- [Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated Distillation](https://mlsanity.com/paper/2204.07028/) (2022)
- [FedICT: Federated Multi-task Distillation for Multi-access Edge Computing](https://mlsanity.com/paper/2301.00389/) (2023)
- [FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence](https://mlsanity.com/paper/2308.07816/) (2023)
- [Agglomerative Federated Learning: Empowering Larger Model Training via End-Edge-Cloud Collaboration](https://mlsanity.com/paper/2312.11489/) (2023)

## Built on by (most cited first, 4 of 4)

- [Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey](https://mlsanity.com/paper/2412.13840/) (2024)
- [FedCache 2.0: Federated Edge Learning with Knowledge Caching and Dataset Distillation](https://mlsanity.com/paper/2405.13378/) (2024)
- [Peak-Controlled Logits Poisoning Attack in Federated Distillation](https://mlsanity.com/paper/2407.18039/) (2024)
- [Onboarding Without Forgetting: Hypernetwork Personalization with Data-Free Replay for Personalized Federated Learning](https://mlsanity.com/paper/2508.05157/) (2025)

## Abstract (arXiv)

Federated Learning (FL) enables collaborative model training among participants while guaranteeing the privacy of raw data. Mainstream FL methodologies overlook the dynamic nature of real-world data, particularly its tendency to grow in volume and diversify in classes over time. This oversight results in FL methods suffering from catastrophic forgetting, where the trained models inadvertently discard previously learned information upon assimilating new data. In response to this challenge, we propose a novel Federated Class-Incremental Learning (FCIL) method, named underlineFederated underlineClass-Incremental underlineLearning with New-Class underlineAugmented underlineSelf-DiunderlineStillation (FedCLASS). The core of FedCLASS is to enrich the class scores of historical models with new class scores predicted by current models and utilize the combined knowledge for self-distillation, enabling a more sufficient and precise knowledge transfer from historical models to current models. Theoretical analyses demonstrate that FedCLASS stands on reliable foundations, considering scores of old classes predicted by historical models as conditional probabilities in the absence of new classes, and the scores of new classes predicted by current models as the conditional probabilities of class scores derived from historical models. Empirical experiments demonstrate the superiority of FedCLASS over four baseline algorithms in reducing average forgetting rate and boosting global accuracy.