# Federated Continual Learning for Text Classification via Selective Inter-client Transfer

arXiv: https://arxiv.org/abs/2210.06101 · HTML: https://mlsanity.com/paper/2210.06101/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 12 Oct 2022.

## Builds on (most cited first, 13 of 13)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [Federated Continual Learning with Weighted Inter-client Transfer](https://mlsanity.com/paper/2003.03196/) (2020)
- [Overcoming Forgetting in Federated Learning on Non-IID Data](https://mlsanity.com/paper/1910.07796/) (2019)
- [Hierarchical Federated Learning Across Heterogeneous Cellular Networks](https://mlsanity.com/paper/1909.02362/) (2019)
- [Federated Unsupervised Representation Learning](https://mlsanity.com/paper/2010.08982/) (2020)
- [In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning](https://mlsanity.com/paper/1809.07857/) (2018)
- [Neural Topic Modeling with Continual Lifelong Learning](https://mlsanity.com/paper/2006.10909/) (2020)

## Built on by (most cited first, 9 of 9)

- [Towards Building the Federated GPT: Federated Instruction Tuning](https://mlsanity.com/paper/2305.05644/) (2023)
- [TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation](https://mlsanity.com/paper/2303.06937/) (2023)
- [Federated Continual Learning via Knowledge Fusion: A Survey](https://mlsanity.com/paper/2312.16475/) (2023)
- [Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning](https://mlsanity.com/paper/2409.01128/) (2024)
- [Federated Continual Learning for Edge-AI: A Comprehensive Survey](https://mlsanity.com/paper/2411.13740/) (2024)
- [Feature Aggregation with Latent Generative Replay for Federated Continual Learning of Socially Appropriate Robot Behaviours](https://mlsanity.com/paper/2405.15773/) (2024)
- [FedNE: Surrogate-Assisted Federated Neighbor Embedding for Dimensionality Reduction](https://mlsanity.com/paper/2409.11509/) (2024)
- [Federated Learning with New Knowledge: Fundamentals, Advances, and Futures](https://mlsanity.com/paper/2402.02268/) (2024)
- [When Generator Replay Degrades: Projected Rehearsal Orchestration for Heterogeneous Federated Class-Incremental Learning](https://mlsanity.com/paper/2606.15695/) (2026)

## Abstract (arXiv)

In this work, we combine the two paradigms: Federated Learning (FL) and Continual Learning (CL) for text classification task in cloud-edge continuum. The objective of Federated Continual Learning (FCL) is to improve deep learning models over life time at each client by (relevant and efficient) knowledge transfer without sharing data. Here, we address challenges in minimizing inter-client interference while knowledge sharing due to heterogeneous tasks across clients in FCL setup. In doing so, we propose a novel framework, Federated Selective Inter-client Transfer (FedSeIT) which selectively combines model parameters of foreign clients. To further maximize knowledge transfer, we assess domain overlap and select informative tasks from the sequence of historical tasks at each foreign client while preserving privacy. Evaluating against the baselines, we show improved performance, a gain of (average) 12.4% in text classification over a sequence of tasks using five datasets from diverse domains. To the best of our knowledge, this is the first work that applies FCL to NLP.