mlsanityData to 14 Oct 2026

Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensions

arXiv 2505.15579 ↗First posted 21 May 2025cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Personalized federated learning has emerged as a popular approach to training on devices holding statistically heterogeneous data, known as clients. However, most existing approaches require a client to have labeled data for training or finetuning in order to obtain their own personalized model. In this paper we address this by proposing FLowDUP, a novel method that is able to generate a personalized model using only a forward pass with unlabeled data. The generated model parameters reside in a low-dimensional subspace, enabling efficient communication and computation. FLowDUP's learning objective is theoretically motivated by our new transductive multi-task PAC-Bayesian generalization bound, that provides performance guarantees for unlabeled clients. The objective is structured in such a way that it allows both clients with labeled data and clients with only unlabeled data to contribute to the training process. To supplement our theoretical results we carry out a thorough experimental evaluation of FLowDUP, demonstrating strong empirical performance on a range of datasets with differing sorts of statistically heterogeneous clients. Through numerous ablation studies, we test the efficacy of the individual components of the method.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 61 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

38 of 61 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2016HyperNetworkscited by 1,027
  7. 2018LEAF: A Benchmark for Federated Settingscited by 792
  8. 2017Federated Multi-Task Learningcited by 672
  9. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  10. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
  11. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  12. 2019Federated Learning with Personalization Layerscited by 541
26 more
  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  2. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  3. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  4. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 370
  5. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  6. 2020Few-Shot Learning via Learning the Representation, Provablycited by 214
  7. 2021Personalized Federated Learning using Hypernetworkscited by 212
  8. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  9. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 123
  10. 2020Federated Unsupervised Representation Learningcited by 93
  11. 2017Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theorycited by 76
  12. 2022PAC-Bayes Compression Bounds So Tight That They Can Explain Generalizationcited by 76
  13. 2013A PAC-Bayesian bound for Lifelong Learningcited by 69
  14. 2021Personalized Federated Learning through Local Memorizationcited by 57
  15. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  16. 2023Personalized Federated Learning under Mixture of Distributionscited by 37
  17. 2019Learning Internal Representations (COLT 1995)cited by 25
  18. 2022FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networkscited by 23
  19. 2023PeFLL: Personalized Federated Learning by Learning to Learncited by 21
  20. 2016Multi-Task Learning with Labeled and Unlabeled Taskscited by 17
  21. 2016Local Rademacher Complexity-based Learning Guarantees for Multi-Task Learningcited by 16
  22. 2022Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practicecited by 11
  23. 2024More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithmscited by 11
  24. 2025From Low Intrinsic Dimensionality to Non-Vacuous Generalization Bounds in Deep Multi-Task Learningcited by 5
  25. 2025ZIP: An Efficient Zeroth-order Prompt Tuning for Black-box Vision-Language Modelscited by 4
  26. 2023UPFL: Unsupervised Personalized Federated Learning towards New Clientscited by 3

Built on by

Not cited yet.