mlsanityData to 7 Oct 2026

Understanding Federated Learning from IID to Non-IID dataset: An Experimental Study

arXiv 2502.00182 ↗First posted 31 Jan 2025cs.LG cs.AI stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

As privacy concerns and data regulations grow, federated learning (FL) has emerged as a promising approach for training machine learning models across decentralized data sources without sharing raw data. However, a significant challenge in FL is that client data are often non-IID (non-independent and identically distributed), leading to reduced performance compared to centralized learning. While many methods have been proposed to address this issue, their underlying mechanisms are often viewed from different perspectives. Through a comprehensive investigation from gradient descent to FL, and from IID to non-IID data settings, we find that inconsistencies in client loss landscapes primarily cause performance degradation in non-IID scenarios. From this understanding, we observe that existing methods can be grouped into two main strategies: (i) adjusting parameter update paths and (ii) modifying client loss landscapes. These findings offer a clear perspective on addressing non-IID challenges in FL and help guide future research in the field.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 18 of its 23 references that are in the map. First posted on arXiv 31 Jan 2025.

Builds on

18 of 23 references.

References · 18

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  5. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  6. 2018Federated Learning with Non-IID Datacited by 1,099
  7. 2020Adaptive Federated Optimizationcited by 1,036
  8. 2016An overview of gradient descent optimization algorithmscited by 1,021
  9. 2021Model-Contrastive Federated Learningcited by 762
  10. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  11. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  12. 2018Horovod: fast and easy distributed deep learning in TensorFlowcited by 275
6 more
  1. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  2. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  3. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  4. 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
  5. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  6. 2023FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learningcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025GC-Fed: Gradient Centralized Federated Learning with Partial Client Participationcited by 2
  2. 2026Federated Inference: Toward Privacy-Preserving Collaborative and Incentivized Model Servingcited by 1
  3. 2025Kuramoto-FedAvg: Using Synchronization Dynamics to Improve Federated Learning Optimization under Statistical Heterogeneitycited by 0