# AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data

arXiv: https://arxiv.org/abs/2505.12245 · HTML: https://mlsanity.com/paper/2505.12245/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 58 references that are in the map.
- First posted on arXiv 18 May 2025.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [Overcoming Catastrophic Forgetting by Incremental Moment Matching](https://mlsanity.com/paper/1703.08475/) (2017)
- [RanPAC: Random Projections and Pre-trained Models for Continual Learning](https://mlsanity.com/paper/2307.02251/) (2023)
- [Federated Continual Learning with Weighted Inter-client Transfer](https://mlsanity.com/paper/2003.03196/) (2020)
- [Continual Learning with Pre-Trained Models: A Survey](https://mlsanity.com/paper/2401.16386/) (2024)
- [Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks](https://mlsanity.com/paper/2006.07322/) (2020)
- [An Empirical Investigation of the Role of Pre-training in Lifelong Learning](https://mlsanity.com/paper/2112.09153/) (2021)
- [Federated Class-Incremental Learning](https://mlsanity.com/paper/2203.11473/) (2022)
- [Preservation of the Global Knowledge by Not-True Distillation in Federated Learning](https://mlsanity.com/paper/2106.03097/) (2021)
- [On the Importance and Applicability of Pre-Training for Federated Learning](https://mlsanity.com/paper/2206.11488/) (2022)
- [Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning](https://mlsanity.com/paper/2210.08090/) (2022)
- [On Loss Functions for Deep Neural Networks in Classification](https://mlsanity.com/paper/1702.05659/) (2017)
- [TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation](https://mlsanity.com/paper/2303.06937/) (2023)
- [ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection](https://mlsanity.com/paper/2205.14922/) (2022)
- [FedFed: Feature Distillation against Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2310.05077/) (2023)
- [Federated Continual Learning via Knowledge Fusion: A Survey](https://mlsanity.com/paper/2312.16475/) (2023)
- [A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Tasks](https://mlsanity.com/paper/2311.07784/) (2023)
- [Non-IID data and Continual Learning processes in Federated Learning: A long road ahead](https://mlsanity.com/paper/2111.13394/) (2021)
- [Online Hyperparameter Optimization for Class-Incremental Learning](https://mlsanity.com/paper/2301.05032/) (2023)
- [Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuning](https://mlsanity.com/paper/2308.09303/) (2023)
- [An Aggregation-Free Federated Learning for Tackling Data Heterogeneity](https://mlsanity.com/paper/2404.18962/) (2024)
- [Personalized Federated Continual Learning via Multi-granularity Prompt](https://mlsanity.com/paper/2407.00113/) (2024)
- [Text-Enhanced Data-free Approach for Federated Class-Incremental Learning](https://mlsanity.com/paper/2403.14101/) (2024)
- [Locality Sensitive Sparse Encoding for Learning World Models Online](https://mlsanity.com/paper/2401.13034/) (2024)
- [AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models](https://mlsanity.com/paper/2405.16240/) (2024)

## Built on by (most cited first, 3 of 3)

- [Exact Federated Continual Unlearning for Ridge Heads on Frozen Foundation Models](https://mlsanity.com/paper/2603.12977/) (2026)
- [DeepAFL: Deep Analytic Federated Learning](https://mlsanity.com/paper/2603.00579/) (2026)
- [Improving Learning of New Diseases through Knowledge-Enhanced Initialization for Federated Adapter Tuning](https://mlsanity.com/paper/2508.10299/) (2025)

## Abstract (arXiv)

Federated Continual Learning (FCL) enables distributed clients to collaboratively train a global model from online task streams in dynamic real-world scenarios. However, existing FCL methods face challenges of both spatial data heterogeneity among distributed clients and temporal data heterogeneity across online tasks. Such data heterogeneity significantly degrades the model performance with severe spatial-temporal catastrophic forgetting of local and past knowledge. In this paper, we identify that the root cause of this issue lies in the inherent vulnerability and sensitivity of gradients to non-IID data. To fundamentally address this issue, we propose a gradient-free method, named Analytic Federated Continual Learning (AFCL), by deriving analytical (i.e., closed-form) solutions from frozen extracted features. In local training, our AFCL enables single-epoch learning with only a lightweight forward-propagation process for each client. In global aggregation, the server can recursively and efficiently update the global model with single-round aggregation. Theoretical analyses validate that our AFCL achieves spatio-temporal invariance of non-IID data. This ideal property implies that, regardless of how heterogeneous the data are distributed across local clients and online tasks, the aggregated model of our AFCL remains invariant and identical to that of centralized joint learning. Extensive experiments show the consistent superiority of our AFCL over state-of-the-art baselines across various benchmark datasets and settings.