mlsanityData to 14 Oct 2026

New Metrics to Evaluate the Performance and Fairness of Personalized Federated Learning

arXiv 2107.13173 ↗First posted 28 Jul 2021cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

In Federated Learning (FL), the clients learn a single global model (FedAvg) through a central aggregator. In this setting, the non-IID distribution of the data across clients restricts the global FL model from delivering good performance on the local data of each client. Personalized FL aims to address this problem by finding a personalized model for each client. Recent works widely report the average personalized model accuracy on a particular data split of a dataset to evaluate the effectiveness of their methods. However, considering the multitude of personalization approaches proposed, it is critical to study the per-user personalized accuracy and the accuracy improvements among users with an equitable notion of fairness. To address these issues, we present a set of performance and fairness metrics intending to assess the quality of personalized FL methods. We apply these metrics to four recently proposed personalized FL methods, PersFL, FedPer, pFedMe, and Per-FedAvg, on three different data splits of the CIFAR-10 dataset. Our evaluations show that the personalized model with the highest average accuracy across users may not necessarily be the fairest. Our code is available at https://tinyurl.com/1hp9ywfa for public use.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 18 of its 24 references that are in the map. First posted on arXiv 28 Jul 2021.

Builds on

18 of 24 references.

References · 18

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  3. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  4. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  5. 2017Federated Multi-Task Learningcited by 672
  6. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  7. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  8. 2019Federated Learning with Personalization Layerscited by 541
  9. 2019Fair Resource Allocation in Federated Learningcited by 394
  10. 2020Adaptive Personalized Federated Learningcited by 358
  11. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  12. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
6 more
  1. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  2. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
  3. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  4. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  5. 2020LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasetscited by 60
  6. 2021Unifying Distillation with Personalization in Federated Learningcited by 3

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 3; 2024: 3; 2025: 2.

11 papers.

  1. 2021Towards Fairness-Aware Federated Learningcited by 58
  2. 2022SemiPFL: Personalized Semi-Supervised Federated Learning Framework for Edge Intelligencecited by 7
  3. 2022PersA-FL: Personalized Asynchronous Federated Learningcited by 5
  4. 2023Mitigating Group Bias in Federated Learning for Heterogeneous Devicescited by 5
  5. 2023Federated Learning for Predictive Maintenance and Quality Inspection in Industrial Applicationscited by 4
  6. 2024Federated Fairness Analytics: Quantifying Fairness in Federated Learningcited by 4
  7. 2024Linkage on Security, Privacy and Fairness in Federated Learning: New Balances and New Perspectivescited by 3
  8. 2023Can We Trust the Similarity Measurement in Federated Learning?cited by 1
  9. 2024Personalized Federated Learning Techniques: Empirical Analysiscited by 1
  10. 2025PLayer-FL: A Principled Approach to Personalized Layer-wise Cross-Silo Federated Learningcited by 1
  11. 2025Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learningcited by 0