mlsanityData to 14 Oct 2026

pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learning

arXiv 2206.03655 ↗First posted 8 Jun 2022cs.LGcited by 44

Not on the map. Why?

Abstract (arXiv)

Personalized Federated Learning (pFL), which utilizes and deploys distinct local models, has gained increasing attention in recent years due to its success in handling the statistical heterogeneity of FL clients. However, standardized evaluation and systematical analysis of diverse pFL methods remain a challenge. Firstly, the highly varied datasets, FL simulation settings and pFL implementations prevent easy and fair comparisons of pFL methods. Secondly, the current pFL literature diverges in the adopted evaluation and ablation protocols. Finally, the effectiveness and robustness of pFL methods are under-explored in various practical scenarios, such as the generalization to new clients and the participation of resource-limited clients. To tackle these challenges, we propose the first comprehensive pFL benchmark, pFL-Bench, for facilitating rapid, reproducible, standardized and thorough pFL evaluation. The proposed benchmark contains more than 10 dataset variants in various application domains with a unified data partition and realistic heterogeneous settings; a modularized and easy-to-extend pFL codebase with more than 20 competitive pFL method implementations; and systematic evaluations under containerized environments in terms of generalization, fairness, system overhead, and convergence. We highlight the benefits and potential of state-of-the-art pFL methods and hope the pFL-Bench enables further pFL research and broad applications that would otherwise be difficult owing to the absence of a dedicated benchmark. The code is released at https://github.com/alibaba/FederatedScope/tree/master/benchmark/pFL-Bench.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Builds on 64 of its 99 references that are in the map. First posted on arXiv 8 Jun 2022.

Builds on

64 of 99 references.

References · 64

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  3. 2018How Powerful are Graph Neural Networks?cited by 4,393
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  7. 2018Federated Learning with Non-IID Datacited by 1,108
  8. 2020Adaptive Federated Optimizationcited by 1,046
  9. 2018Neural Network Acceptability Judgmentscited by 958
  10. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  11. 2019Towards Federated Learning at Scale: System Designcited by 832
  12. 2018LEAF: A Benchmark for Federated Settingscited by 792
38 more
  1. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  2. 2017Federated Multi-Task Learningcited by 672
  3. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  4. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  5. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  6. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  7. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  8. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
  9. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  10. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  11. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  12. 2021Towards Personalized Federated Learningcited by 464
  13. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 455
  14. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  15. 2020Adaptive Personalized Federated Learningcited by 358
  16. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  17. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  18. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  19. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  20. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 300
  21. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
  22. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  23. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
  24. 2021Personalized Federated Learning using Hypernetworkscited by 212
  25. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  26. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 204
  27. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  28. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  29. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 190
  30. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  31. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
  32. 2020Threats to Federated Learning: A Surveycited by 178
  33. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 175
  34. 2020Survey of Personalization Techniques for Federated Learningcited by 145
  35. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 136
  36. 2020FedVision: An Online Visual Object Detection Platform Powered by Federated Learningcited by 124
  37. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 123
  38. 2021Federated Graph Classification over Non-IID Graphscited by 114

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 15; 2024: 15; 2025: 7; 2026: 5.

44 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  2. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 149
  3. 2024Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resourcescited by 93
  4. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  5. 2022Motley: Benchmarking Heterogeneity and Personalization in Federated Learningcited by 31
  6. 2023Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learningcited by 29
  7. 2023Privacy and Fairness in Federated Learning: on the Perspective of Trade-offcited by 26
  8. 2023FS-Real: Towards Real-World Cross-Device Federated Learningcited by 22
  9. 2023Revisiting Personalized Federated Learning: Robustness Against Backdoor Attackscited by 16
  10. 2023Benchmarking Algorithms for Federated Domain Generalizationcited by 14
  11. 2024Federated Learning Priorities Under the European Union Artificial Intelligence Actcited by 13
  12. 2023Profit: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuningcited by 9
32 more
  1. 2024Spectral Co-Distillation for Personalized Federated Learningcited by 8
  2. 2024pfl-research: simulation framework for accelerating research in Private Federated Learningcited by 8
  3. 2024COALA: A Practical and Vision-Centric Federated Learning Platformcited by 7
  4. 2023FedAds: A Benchmark for Privacy-Preserving CVR Estimation with Vertical Federated Learningcited by 6
  5. 2024Why Go Full? Elevating Federated Learning Through Partial Network Updatescited by 6
  6. 2025Federated Learning for Medical Image Classification: A Comprehensive Benchmarkcited by 6
  7. 2023A Practical Recipe for Federated Learning Under Statistical Heterogeneity Experimental Designcited by 5
  8. 2024Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approachcited by 5
  9. 2022Collaborating Heterogeneous Natural Language Processing Tasks via Federated Learningcited by 4
  10. 2024Formal Logic Enabled Personalized Federated Learning Through Property Inferencecited by 4
  11. 2024FedRSU: Federated Learning for Scene Flow Estimation on Roadside Unitscited by 2
  12. 2024Overcoming label shift with target-aware federated learningcited by 2
  13. 2025Bad-PFL: Exploring Backdoor Attacks against Personalized Federated Learningcited by 2
  14. 2023HPN: Personalized Federated Hyperparameter Optimizationcited by 1
  15. 2023Personalized Federated Learning via Gradient Modulation for Heterogeneous Text Summarizationcited by 1
  16. 2023A Comprehensive View of Personalized Federated Learning on Heterogeneous Clinical Datasetscited by 1
  17. 2024A Note on Bias to Completecited by 1
  18. 2025FedMeNF: Privacy-Preserving Federated Meta-Learning for Neural Fieldscited by 1
  19. 2025Federated Learning via Meta-Variational Dropoutcited by 1
  20. 2026FedXDS: Leveraging Model Attribution Methods to counteract Data Heterogeneity in Federated Learningcited by 1
  21. 2023FedL2P: Federated Learning to Personalizecited by 0
  22. 2024OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learningcited by 0
  23. 2024DA-PFL: Dynamic Affinity Aggregation for Personalized Federated Learningcited by 0
  24. 2024Personalized Multi-tier Federated Learningcited by 0
  25. 2024A cautionary tale on the cost-effectiveness of collaborative AI in real-world medical applicationscited by 0
  26. 2025FLIP: Towards Comprehensive and Reliable Evaluation of Federated Prompt Learningcited by 0
  27. 2025Adaptive Latent-Space Constraints in Personalized Federated Learningcited by 0
  28. 2025Local Performance vs. Out-of-Distribution Generalization: An Empirical Analysis of Personalized Federated Learning in Heterogeneous Data Environmentscited by 0
  29. 2026SA-PEF: Step-Ahead Partial Error Feedback for Efficient Federated Learningcited by 0
  30. 2026Social Hippocampus Memory Learningcited by 0
  31. 2026Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deploymentcited by 0
  32. 2026FedLTLib: A Comprehensive Benchmark for Federated Long-Tail Learningcited by 0