mlsanityData to 7 Oct 2026

Lazy But Effective: Collaborative Personalized Federated Learning with Heterogeneous Data

arXiv 2505.02540 ↗First posted 5 May 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In Federated Learning, heterogeneity in client data distributions often means that a single global model does not have the best performance for individual clients. Consider for example training a next-word prediction model for keyboards: user-specific language patterns due to demographics (dialect, age, etc.), language proficiency, and writing style result in a highly non-IID dataset across clients. Other examples are medical images taken with different machines, or driving data from different vehicle types. To address this, we propose a simple yet effective personalized federated learning framework (pFedLIA) that utilizes a computationally efficient influence approximation, called `Lazy Influence', to cluster clients in a distributed manner before model aggregation. Within each cluster, data owners collaborate to jointly train a model that captures the specific data patterns of the clients. Our method has been shown to successfully recover the global model's performance drop due to the non-IID-ness in various synthetic and real-world settings, specifically a next-word prediction task on the Nordic languages as well as several benchmark tasks. It matches the performance of a hypothetical Oracle clustering, and significantly improves on existing baselines, e.g., an improvement of 17% on CIFAR100.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 47 of its 64 references that are in the map. First posted on arXiv 5 May 2025.

Builds on

47 of 64 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  7. 2018Federated Learning with Non-IID Datacited by 1,099
  8. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  9. 2018LEAF: A Benchmark for Federated Settingscited by 780
  10. 2017Federated Multi-Task Learningcited by 669
  11. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  12. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
35 more
  1. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  2. 2019Federated Learning with Personalization Layerscited by 533
  3. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  4. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  5. 2021Towards Personalized Federated Learningcited by 462
  6. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  7. 2019Federated Machine Learning: Concept and Applicationscited by 409
  8. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  9. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  10. 2020Adaptive Personalized Federated Learningcited by 353
  11. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  12. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  13. 2021A Field Guide to Federated Optimizationcited by 312
  14. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  15. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  16. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  17. 2021Personalized Federated Learning using Hypernetworkscited by 210
  18. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  19. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  20. 2019A Survey on Distributed Machine Learningcited by 187
  21. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
  22. 2020A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languagescited by 131
  23. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  24. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  25. 2022Layer-wised Model Aggregation for Personalized Federated Learningcited by 95
  26. 2017Personalized and Private Peer-to-Peer Machine Learningcited by 54
  27. 2019Federated Learning Of Out-Of-Vocabulary Wordscited by 51
  28. 2021Flexible Clustered Federated Learning for Client-Level Data Distribution Shiftcited by 47
  29. 2022MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcarecited by 36
  30. 2020Dynamic Clustering in Federated Learningcited by 27
  31. 2023PeFLL: Personalized Federated Learning by Learning to Learncited by 21
  32. 2020Toward Understanding the Influence of Individual Clients in Federated Learningcited by 16
  33. 2022FedREP: Towards Horizontal Federated Load Forecasting for Retail Energy Providerscited by 14
  34. 2023pFedSim: Similarity-Aware Model Aggregation Towards Personalized Federated Learningcited by 9
  35. 2022LIA: Privacy-Preserving Data Quality Evaluation in Federated Learning Using a Lazy Influence Approximationcited by 3

Built on by

Not cited yet.