# Onboarding Without Forgetting: Hypernetwork Personalization with Data-Free Replay for Personalized Federated Learning

arXiv: https://arxiv.org/abs/2508.05157 · HTML: https://mlsanity.com/paper/2508.05157/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 7 Aug 2025.

## Builds on (most cited first, 17 of 17)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [Federated Learning with Personalization Layers](https://mlsanity.com/paper/1912.00818/) (2019)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion](https://mlsanity.com/paper/1912.08795/) (2019)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [Personalized Federated Learning: A Meta-Learning Approach](https://mlsanity.com/paper/2002.07948/) (2020)
- [Personalized Federated Learning using Hypernetworks](https://mlsanity.com/paper/2103.04628/) (2021)
- [Federated Continual Learning with Weighted Inter-client Transfer](https://mlsanity.com/paper/2003.03196/) (2020)
- [Federated Continual Learning via Knowledge Fusion: A Survey](https://mlsanity.com/paper/2312.16475/) (2023)
- [FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuning](https://mlsanity.com/paper/2404.02478/) (2024)
- [PeFLL: Personalized Federated Learning by Learning to Learn](https://mlsanity.com/paper/2306.05515/) (2023)
- [Federated Continual Learning for Edge-AI: A Comprehensive Survey](https://mlsanity.com/paper/2411.13740/) (2024)
- [Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgetting](https://mlsanity.com/paper/2501.15356/) (2025)
- [Federated Class-Incremental Learning with New-Class Augmented Self-Distillation](https://mlsanity.com/paper/2401.00622/) (2024)

## Abstract (arXiv)

Federated Learning (FL) enables collaborative training across distributed clients without sharing raw data, offering strong privacy benefits. However, most methods assume all clients remain available throughout training, which is unrealistic as new clients often join over time. We study this setting, where the task and label space stay fixed but clients arrive in batches. Our analysis reveals two key challenges: updating the shared model only with new clients harms existing clients, while freezing it protects them but blocks gains from new knowledge. To capture these trade-offs, we introduce Proactive Adaptation (PA) for onboarding gains and Retroactive Improvement (RI) for changes in earlier clients without retraining. We then propose pFedDSH, which combines a central hypernetwork for personalized initialization, batch-specific binary masks for capacity preservation and allocation, and server-side data-free replay to propagate improvements without exposing client data. Experiments show that pFedDSH preserves stability for existing clients while keeping communication and adaptation costs unchanged for new clients.