mlsanityData to 7 Oct 2026

The Key of Parameter Skew in Federated Learning

arXiv 2408.11278 ↗First posted 21 Aug 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has emerged as an excellent solution for performing deep learning on different data owners without exchanging raw data. However, statistical heterogeneity in FL presents a key challenge, leading to a phenomenon of skewness in local model parameter distributions that researchers have largely overlooked. In this work, we propose the concept of parameter skew to describe the phenomenon that can substantially affect the accuracy of global model parameter estimation. Additionally, we introduce FedSA, an aggregation strategy to obtain a high-quality global model, to address the implication from parameter skew. Specifically, we categorize parameters into high-dispersion and low-dispersion groups based on the coefficient of variation. For high-dispersion parameters, Micro-Classes (MIC) and Macro-Classes (MAC) represent the dispersion at the micro and macro levels, respectively, forming the foundation of FedSA. To evaluate the effectiveness of FedSA, we conduct extensive experiments with different FL algorithms on three computer vision datasets. FedSA outperforms eight state-of-the-art baselines by about 4.7% in test accuracy.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 31 references that are in the map. First posted on arXiv 21 Aug 2024.

Builds on

23 of 31 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2021Model-Contrastive Federated Learningcited by 762
  6. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  7. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  8. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  9. 2020Federated Learning with Matched Averagingcited by 564
  10. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  11. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
  12. 2021Personalized Federated Learning using Hypernetworkscited by 210
11 more
  1. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
  2. 2020FedVision: An Online Visual Object Detection Platform Powered by Federated Learningcited by 121
  3. 2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 121
  4. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  5. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  6. 2020FedHome: Cloud-Edge based Personalized Federated Learning for In-Home Health Monitoringcited by 61
  7. 2021FLOP: Federated Learning on Medical Datasets using Partial Networkscited by 42
  8. 2022AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimationcited by 21
  9. 2023Exploiting Label Skews in Federated Learning with Model Concatenationcited by 14
  10. 2022FedGen: Generalizable Federated Learning for Sequential Datacited by 9
  11. 2022Towards Federated Long-Tailed Learningcited by 7

Built on by

Not cited yet.