mlsanityData to 7 Oct 2026

Feature Norm Regularized Federated Learning: Transforming Skewed Distributions into Global Insights

arXiv 2312.06951 ↗First posted 12 Dec 2023cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

In the field of federated learning, addressing non-independent and identically distributed (non-i.i.d.) data remains a quintessential challenge for improving global model performance. This work introduces the Feature Norm Regularized Federated Learning (FNR-FL) algorithm, which uniquely incorporates class average feature norms to enhance model accuracy and convergence in non-i.i.d. scenarios. Our comprehensive analysis reveals that FNR-FL not only accelerates convergence but also significantly surpasses other contemporary federated learning algorithms in test accuracy, particularly under feature distribution skew scenarios. The novel modular design of FNR-FL facilitates seamless integration with existing federated learning frameworks, reinforcing its adaptability and potential for widespread application. We substantiate our claims through rigorous empirical evaluations, demonstrating FNR-FL's exceptional performance across various skewed data distributions. Relative to FedAvg, FNR-FL exhibits a substantial 66.24% improvement in accuracy and a significant 11.40% reduction in training time, underscoring its enhanced effectiveness and efficiency.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 12 Dec 2023.

Builds on

26 of 37 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  5. 2018Federated Learning with Non-IID Datacited by 1,099
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  7. 2021Model-Contrastive Federated Learningcited by 762
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  9. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  10. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  11. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 555
14 more
  1. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  2. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  3. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  4. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  5. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  6. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 150
  7. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  8. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  9. 2018On the Fenchel Duality between Strong Convexity and Lipschitz Continuous Gradientcited by 42
  10. 2023A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiencycited by 21
  11. 2022Rethinking Normalization Methods in Federated Learningcited by 19
  12. 2022Fast Composite Optimization and Statistical Recovery in Federated Learningcited by 12
  13. 2023FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learningcited by 5
  14. 2023Heterogeneous Datasets for Federated Survival Analysis Simulationcited by 4

Built on by

Not cited yet.