Widening the Network Mitigates the Impact of Data Heterogeneity on FedAvg
Not on the map. Why?
Abstract (arXiv)
Federated learning (FL) enables decentralized clients to train a model collaboratively without sharing local data. A key distinction between FL and centralized learning is that clients' data are non-independent and identically distributed, which poses significant challenges in training a global model that generalizes well across heterogeneous local data distributions. In this paper, we analyze the convergence of overparameterized FedAvg with gradient descent (GD). We prove that the impact of data heterogeneity diminishes as the width of neural networks increases, ultimately vanishing when the width approaches infinity. In the infinite-width regime, we further prove that both the global and local models in FedAvg behave as linear models, and that FedAvg achieves the same generalization performance as centralized learning with the same number of GD iterations. Extensive experiments validate our theoretical findings across various network architectures, loss functions, and optimization methods.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 36 references that are in the map. First posted on arXiv 18 Aug 2025.
Builds on
28 of 36 references.
References · 28
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2016Wide Residual Networkscited by 3,070
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
- 2018Federated Learning with Non-IID Datacited by 1,108
- 2020Adaptive Federated Optimizationcited by 1,046
- 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
- 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
- 2021Model-Contrastive Federated Learningcited by 766
- 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
- 2020Personalized Federated Learning with Moreau Envelopescited by 618
- 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
16 more
- 2020Federated Learning with Matched Averagingcited by 564
- 2021Federated Learning Based on Dynamic Regularizationcited by 562
- 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 342
- 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 198
- 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 144
- 2018Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networkscited by 142
- 2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 101
- 2019First Analysis of Local GD on Heterogeneous Datacited by 81
- 2019Active Federated Learningcited by 63
- 2022Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reductioncited by 32
- 2022TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernelscited by 26
- 2021Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learningcited by 23
- 2019The asymptotic spectrum of the Hessian of DNN throughout trainingcited by 19
- 2019The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Studycited by 16
- 2021Neural Tangent Kernel Empowered Federated Learningcited by 15
- 2021Local SGD Optimizes Overparameterized Neural Networks in Polynomial Timecited by 10
Built on by
Citing papers per year: 2026: 2.
2 papers.