# FedET: A Communication-Efficient Federated Class-Incremental Learning Framework Based on Enhanced Transformer

arXiv: https://arxiv.org/abs/2306.15347 · HTML: https://mlsanity.com/paper/2306.15347/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 27 Jun 2023.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Do Deep Nets Really Need to be Deep?](https://mlsanity.com/paper/1312.6184/) (2013)
- [AdapterDrop: On the Efficiency of Adapters in Transformers](https://mlsanity.com/paper/2010.11918/) (2020)
- [Mnemonics Training: Multi-Class Incremental Learning without Forgetting](https://mlsanity.com/paper/2002.10211/) (2020)
- [Incremental Learning of Object Detectors without Catastrophic Forgetting](https://mlsanity.com/paper/1708.06977/) (2017)
- [Episodic Memory in Lifelong Language Learning](https://mlsanity.com/paper/1906.01076/) (2019)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [Class-incremental Learning via Deep Model Consolidation](https://mlsanity.com/paper/1903.07864/) (2019)
- [Federated Continual Learning with Weighted Inter-client Transfer](https://mlsanity.com/paper/2003.03196/) (2020)
- [Federated Class-Incremental Learning](https://mlsanity.com/paper/2203.11473/) (2022)
- [SS-IL: Separated Softmax for Incremental Learning](https://mlsanity.com/paper/2003.13947/) (2020)
- [Distilling Causal Effect of Data in Class-Incremental Learning](https://mlsanity.com/paper/2103.01737/) (2021)
- [Continual Learning for Text Classification with Information Disentanglement Based Regularization](https://mlsanity.com/paper/2104.05489/) (2021)
- [On Learning the Geodesic Path for Incremental Learning](https://mlsanity.com/paper/2104.08572/) (2021)
- [Learning and Evaluating General Linguistic Intelligence](https://mlsanity.com/paper/1901.11373/) (2019)
- [No One Left Behind: Real-World Federated Class-Incremental Learning](https://mlsanity.com/paper/2302.00903/) (2023)
- [Federated Reconnaissance: Efficient, Distributed, Class-Incremental Learning](https://mlsanity.com/paper/2109.00150/) (2021)
- [Federated Learning with Dynamic Transformer for Text to Speech](https://mlsanity.com/paper/2107.08795/) (2021)

## Built on by (most cited first, 23 of 23)

- [Grounding Foundation Models through Federated Transfer Learning: A General Framework](https://mlsanity.com/paper/2311.17431/) (2023)
- [PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learning](https://mlsanity.com/paper/2401.02094/) (2024)
- [Federated Distillation: A Survey](https://mlsanity.com/paper/2404.08564/) (2024)
- [Rehearsal-Free Continual Federated Learning with Synergistic Synaptic Intelligence](https://mlsanity.com/paper/2412.13779/) (2024)
- [Accurate Forgetting for Heterogeneous Federated Continual Learning](https://mlsanity.com/paper/2502.14205/) (2025)
- [Federated Class-Incremental Learning with Hierarchical Generative Prototypes](https://mlsanity.com/paper/2406.02447/) (2024)
- [Resource-Constrained Federated Continual Learning: What Does Matter?](https://mlsanity.com/paper/2501.08737/) (2025)
- [Federated Class-Incremental Learning with New-Class Augmented Self-Distillation](https://mlsanity.com/paper/2401.00622/) (2024)
- [INCPrompt: Task-Aware incremental Prompting for Rehearsal-Free Class-incremental Learning](https://mlsanity.com/paper/2401.11667/) (2024)
- [Data-Free Federated Class Incremental Learning with Diffusion-Based Generative Memory](https://mlsanity.com/paper/2405.17457/) (2024)
- [P2DT: Mitigating Forgetting in task-incremental Learning with progressive prompt Decision Transformer](https://mlsanity.com/paper/2401.11666/) (2024)
- [FPPL: An Efficient and Non-IID Robust Federated Continual Learning Framework](https://mlsanity.com/paper/2411.01904/) (2024)
- [EdgeMA: Model Adaptation System for Real-Time Video Analytics on Edge Devices](https://mlsanity.com/paper/2308.08717/) (2023)
- [Task-agnostic Decision Transformer for Multi-type Agent Control with Federated Split Training](https://mlsanity.com/paper/2405.13445/) (2024)
- [Task-agnostic Low-rank Residual Adaptation for Efficient Federated Continual Fine-Tuning](https://mlsanity.com/paper/2505.12318/) (2025)
- [Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection](https://mlsanity.com/paper/2509.21606/) (2025)
- [WHALE-FL: Wireless and Heterogeneity Aware Latency Efficient Federated Learning over Mobile Devices via Adaptive Subnetwork Scheduling](https://mlsanity.com/paper/2405.00885/) (2024)
- [Federated Hierarchical Clustering with Automatic Selection of Optimal Cluster Numbers](https://mlsanity.com/paper/2603.12684/) (2026)
- [Optimized Federated Knowledge Distillation with Distributed Neural Architecture Search](https://mlsanity.com/paper/2605.21322/) (2026)
- [When Generator Replay Degrades: Projected Rehearsal Orchestration for Heterogeneous Federated Class-Incremental Learning](https://mlsanity.com/paper/2606.15695/) (2026)
- [Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem](https://mlsanity.com/paper/2608.21539/) (2026)
- [MECAIL: Communication-Aware Incremental Learning for Object Detection with 14.6 KB Spatiotemporal Experts](https://mlsanity.com/paper/2609.24455/) (2026)
- [ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learning](https://mlsanity.com/paper/2609.38833/) (2026)

## Abstract (arXiv)

Federated Learning (FL) has been widely concerned for it enables decentralized learning while ensuring data privacy. However, most existing methods unrealistically assume that the classes encountered by local clients are fixed over time. After learning new classes, this assumption will make the model's catastrophic forgetting of old classes significantly severe. Moreover, due to the limitation of communication cost, it is challenging to use large-scale models in FL, which will affect the prediction accuracy. To address these challenges, we propose a novel framework, Federated Enhanced Transformer (FedET), which simultaneously achieves high accuracy and low communication cost. Specifically, FedET uses Enhancer, a tiny module, to absorb and communicate new knowledge, and applies pre-trained Transformers combined with different Enhancers to ensure high precision on various tasks. To address local forgetting caused by new classes of new tasks and global forgetting brought by non-i.i.d (non-independent and identically distributed) class imbalance across different local clients, we proposed an Enhancer distillation method to modify the imbalance between old and new knowledge and repair the non-i.i.d. problem. Experimental results demonstrate that FedET's average accuracy on representative benchmark datasets is 14.1% higher than the state-of-the-art method, while FedET saves 90% of the communication cost compared to the previous method.