# Unsupervised Model Personalization while Preserving Privacy and Scalability: An Open Problem

arXiv: https://arxiv.org/abs/2003.13296 · HTML: https://mlsanity.com/paper/2003.13296/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2020.

## Built on by (most cited first, 13 of 13)

- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [Federated Class-Incremental Learning](https://mlsanity.com/paper/2203.11473/) (2022)
- [Rehearsal revealed: The limits and merits of revisiting samples in continual learning](https://mlsanity.com/paper/2104.07446/) (2021)
- [Federated Incremental Semantic Segmentation](https://mlsanity.com/paper/2304.04620/) (2023)
- [How Well Does Self-Supervised Pre-Training Perform with Streaming Data?](https://mlsanity.com/paper/2104.12081/) (2021)
- [Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learning](https://mlsanity.com/paper/2212.14284/) (2022)
- [AdaXpert: Adapting Neural Architecture for Growing Data](https://mlsanity.com/paper/2107.00254/) (2021)
- [Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgetting](https://mlsanity.com/paper/2501.15356/) (2025)
- [EgoAdapt: A multi-stream evaluation study of adaptation to real-world egocentric user video](https://mlsanity.com/paper/2307.05784/) (2023)
- [CALA: A Class-Aware Logit Adapter for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2412.12654/) (2024)
- [Out-of-distribution Few-shot Learning For Edge Devices without Model Fine-tuning](https://mlsanity.com/paper/2304.06309/) (2023)
- [A Methodology-Oriented Study of Catastrophic Forgetting in Incremental Deep Neural Networks](https://mlsanity.com/paper/2405.08015/) (2024)
- [Dynamic Textual Prompt For Rehearsal-free Lifelong Person Re-identification](https://mlsanity.com/paper/2411.06023/) (2024)

## Abstract (arXiv)

This work investigates the task of unsupervised model personalization, adapted to continually evolving, unlabeled local user images. We consider the practical scenario where a high capacity server interacts with a myriad of resource-limited edge devices, imposing strong requirements on scalability and local data privacy. We aim to address this challenge within the continual learning paradigm and provide a novel Dual User-Adaptation framework (DUA) to explore the problem. This framework flexibly disentangles user-adaptation into model personalization on the server and local data regularization on the user device, with desirable properties regarding scalability and privacy constraints. First, on the server, we introduce incremental learning of task-specific expert models, subsequently aggregated using a concealed unsupervised user prior. Aggregation avoids retraining, whereas the user prior conceals sensitive raw user data, and grants unsupervised adaptation. Second, local user-adaptation incorporates a domain adaptation point of view, adapting regularizing batch normalization parameters to the user data. We explore various empirical user configurations with different priors in categories and a tenfold of transforms for MIT Indoor Scene recognition, and classify numbers in a combined MNIST and SVHN setup. Extensive experiments yield promising results for data-driven local adaptation and elicit user priors for server adaptation to depend on the model rather than user data. Hence, although user-adaptation remains a challenging open problem, the DUA framework formalizes a principled foundation for personalizing both on server and user device, while maintaining privacy and scalability.