# A Comparative Evaluation of FedAvg and Per-FedAvg Algorithms for Dirichlet Distributed Heterogeneous Data

arXiv: https://arxiv.org/abs/2309.01275 · HTML: https://mlsanity.com/paper/2309.01275/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 3 Sep 2023.

## Builds on (most cited first, 11 of 11)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [Data-Free Knowledge Distillation for Heterogeneous Federated Learning](https://mlsanity.com/paper/2105.10056/) (2021)
- [SCAFFOLD: Stochastic Controlled Averaging for Federated Learning](https://mlsanity.com/paper/1910.06378/) (2019)
- [Privacy and Robustness in Federated Learning: Attacks and Defenses](https://mlsanity.com/paper/2012.06337/) (2020)
- [Federated Learning for Wireless Communications: Motivation, Opportunities and Challenges](https://mlsanity.com/paper/1908.06847/) (2019)
- [Federated Learning with Label Distribution Skew via Logits Calibration](https://mlsanity.com/paper/2209.00189/) (2022)
- [Federated Learning for Water Consumption Forecasting in Smart Cities](https://mlsanity.com/paper/2301.13036/) (2023)

## Built on by (most cited first, 14 of 14)

- [Exploring Vacant Classes in Label-Skewed Federated Learning](https://mlsanity.com/paper/2401.02329/) (2024)
- [FedCFA: Alleviating Simpson's Paradox in Model Aggregation with Counterfactual Federated Learning](https://mlsanity.com/paper/2412.18904/) (2024)
- [A Robust Pipeline for Differentially Private Federated Learning on Imbalanced Clinical Data using SMOTETomek and FedProx](https://mlsanity.com/paper/2508.10017/) (2025)
- [Efficient Collaborations through Weight-Driven Coalition Dynamics in Federated Learning Systems](https://mlsanity.com/paper/2401.12356/) (2024)
- [Effectiveness of Distributed Gradient Descent with Local Steps for Overparameterized Models](https://mlsanity.com/paper/2412.07971/) (2024)
- [FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection](https://mlsanity.com/paper/2604.24012/) (2026)
- [Self-Play Enhancement via Advantage-Weighted Refinement in Online Federated LLM Fine-Tuning with Real-Time Feedback](https://mlsanity.com/paper/2605.07977/) (2026)
- [Centralized vs. Federated Learning for Educational Data Mining: A Comparative Study on Student Performance Prediction with SAEB Microdata](https://mlsanity.com/paper/2509.00086/) (2025)
- [Privacy-Preserving Personalization in Education: A Federated Recommender System for Student Performance Prediction](https://mlsanity.com/paper/2509.10516/) (2025)
- [Hierarchical Federated Learning for Crop Yield Prediction in Smart Agricultural Production Systems](https://mlsanity.com/paper/2510.12727/) (2025)
- [Local Performance vs. Out-of-Distribution Generalization: An Empirical Analysis of Personalized Federated Learning in Heterogeneous Data Environments](https://mlsanity.com/paper/2510.24503/) (2025)
- [FedEMA-Distill: Exponential Moving Average Guided Knowledge Distillation for Robust Federated Learning](https://mlsanity.com/paper/2603.04422/) (2026)
- [FedAgain: A Trust-Based and Robust Federated Learning Strategy for an Automated Kidney Stone Identification in Ureteroscopy](https://mlsanity.com/paper/2603.19512/) (2026)
- [Opinion Dynamics-based Coalition Formation for Federated Learning in Heterogeneous IoT Systems](https://mlsanity.com/paper/2609.19695/) (2026)

## Abstract (arXiv)

In this paper, we investigate Federated Learning (FL), a paradigm of machine learning that allows for decentralized model training on devices without sharing raw data, there by preserving data privacy. In particular, we compare two strategies within this paradigm: Federated Averaging (FedAvg) and Personalized Federated Averaging (Per-FedAvg), focusing on their performance with Non-Identically and Independently Distributed (Non-IID) data. Our analysis shows that the level of data heterogeneity, modeled using a Dirichlet distribution, significantly affects the performance of both strategies, with Per-FedAvg showing superior robustness in conditions of high heterogeneity. Our results provide insights into the development of more effective and efficient machine learning strategies in a decentralized setting.