# Federated User Representation Learning

arXiv: https://arxiv.org/abs/1909.12535 · HTML: https://mlsanity.com/paper/1909.12535/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2019.

## Built on by (most cited first, 30 of 32)

- [Towards Personalized Federated Learning](https://mlsanity.com/paper/2103.00710/) (2021)
- [No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data](https://mlsanity.com/paper/2106.05001/) (2021)
- [Federated Learning with Partial Model Personalization](https://mlsanity.com/paper/2204.03809/) (2022)
- [Federated Reinforcement Learning with Environment Heterogeneity](https://mlsanity.com/paper/2204.02634/) (2022)
- [FederatedScope: A Flexible Federated Learning Platform for Heterogeneity](https://mlsanity.com/paper/2204.05011/) (2022)
- [A Survey on Heterogeneous Federated Learning](https://mlsanity.com/paper/2210.04505/) (2022)
- [PeFLL: Personalized Federated Learning by Learning to Learn](https://mlsanity.com/paper/2306.05515/) (2023)
- [SphereFed: Hyperspherical Federated Learning](https://mlsanity.com/paper/2207.09413/) (2022)
- [On Bridging Generic and Personalized Federated Learning for Image Classification](https://mlsanity.com/paper/2107.00778/) (2021)
- [FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systems](https://mlsanity.com/paper/2304.01534/) (2023)
- [Training Mixed-Domain Translation Models via Federated Learning](https://mlsanity.com/paper/2205.01557/) (2022)
- [Federated Few-shot Learning](https://mlsanity.com/paper/2306.10234/) (2023)
- [Analysis of Privacy Leakage in Federated Large Language Models](https://mlsanity.com/paper/2403.04784/) (2024)
- [Personalized Federated Learning for Spatio-Temporal Forecasting: A Dual Semantic Alignment-Based Contrastive Approach](https://mlsanity.com/paper/2404.03702/) (2024)
- [Federated Learning of Shareable Bases for Personalization-Friendly Image Classification](https://mlsanity.com/paper/2304.07882/) (2023)
- [Federated Natural Language Generation for Personalized Dialogue System](https://mlsanity.com/paper/2110.06419/) (2021)
- [Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning](https://mlsanity.com/paper/2305.13170/) (2023)
- [Aggregate or Not? Exploring Where to Privatize in DNN Based Federated Learning Under Different Non-IID Scenes](https://mlsanity.com/paper/2107.11954/) (2021)
- [Exploiting Personalized Invariance for Better Out-of-distribution Generalization in Federated Learning](https://mlsanity.com/paper/2211.11243/) (2022)
- [UNIDEAL: Curriculum Knowledge Distillation Federated Learning](https://mlsanity.com/paper/2309.08961/) (2023)
- [Incremental user embedding modeling for personalized text classification](https://mlsanity.com/paper/2202.06369/) (2022)
- [PaDPaF: Partial Disentanglement with Partially-Federated GANs](https://mlsanity.com/paper/2212.03836/) (2022)
- [FedSheafHN: Personalized Federated Learning on Graph-structured Data](https://mlsanity.com/paper/2405.16056/) (2024)
- [pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixup](https://mlsanity.com/paper/2501.11002/) (2025)
- [Preliminary Steps Towards Federated Sentiment Classification](https://mlsanity.com/paper/2107.11956/) (2021)
- [Medical Federated Model with Mixture of Personalized and Sharing Components](https://mlsanity.com/paper/2306.14483/) (2023)
- [FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2412.04416/) (2024)
- [Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregation](https://mlsanity.com/paper/2505.14161/) (2025)
- [Unsupervised Federated Optimization at the Edge: D2D-Enabled Learning without Labels](https://mlsanity.com/paper/2404.09861/) (2024)
- [FedCova: Robust Federated Covariance Learning Against Noisy Labels](https://mlsanity.com/paper/2603.04062/) (2026)

## Abstract (arXiv)

Collaborative personalization, such as through learned user representations (embeddings), can improve the prediction accuracy of neural-network-based models significantly. We propose Federated User Representation Learning (FURL), a simple, scalable, privacy-preserving and resource-efficient way to utilize existing neural personalization techniques in the Federated Learning (FL) setting. FURL divides model parameters into federated and private parameters. Private parameters, such as private user embeddings, are trained locally, but unlike federated parameters, they are not transferred to or averaged on the server. We show theoretically that this parameter split does not affect training for most model personalization approaches. Storing user embeddings locally not only preserves user privacy, but also improves memory locality of personalization compared to on-server training. We evaluate FURL on two datasets, demonstrating a significant improvement in model quality with 8% and 51% performance increases, and approximately the same level of performance as centralized training with only 0% and 4% reductions. Furthermore, we show that user embeddings learned in FL and the centralized setting have a very similar structure, indicating that FURL can learn collaboratively through the shared parameters while preserving user privacy.