mlsanityData to 7 Oct 2026

Statistical Estimation and Inference via Local SGD in Federated Learning

arXiv 2109.01326 ↗First posted 3 Sep 2021stat.ML cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) makes a large amount of edge computing devices (e.g., mobile phones) jointly learn a global model without data sharing. In FL, data are generated in a decentralized manner with high heterogeneity. This paper studies how to perform statistical estimation and inference in the federated setting. We analyze the so-called Local SGD, a multi-round estimation procedure that uses intermittent communication to improve communication efficiency. We first establish a it functional central limit theorem that shows the averaged iterates of Local SGD weakly converge to a rescaled Brownian motion. We next provide two iterative inference methods: the it plug-in and the it random scaling. Random scaling constructs an asymptotically pivotal statistic for inference by using the information along the whole Local SGD path. Both the methods are communication efficient and applicable to online data. Our theoretical and empirical results show that Local SGD simultaneously achieves both statistical efficiency and communication efficiency.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 33 of its 58 references that are in the map. First posted on arXiv 3 Sep 2021.

Builds on

33 of 58 references.

References · 33

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2018Federated Learning with Non-IID Datacited by 1,099
  7. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  8. 2018Sparsified SGD with Memorycited by 373
  9. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  10. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  11. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  12. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
21 more
  1. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  2. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  3. 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
  4. 2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
  5. 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 106
  6. 2020Federated Accelerated Stochastic Gradient Descentcited by 85
  7. 2019Variance Reduced Local SGD with Lower Communication Complexitycited by 80
  8. 2016Statistical Inference for Model Parameters in Stochastic Gradient Descentcited by 71
  9. 2015Perturbed Iterate Analysis for Asynchronous Stochastic Optimizationcited by 67
  10. 2020On Linear Stochastic Approximation: Fine-grained Polyak-Ruppert and Non-Asymptotic Concentrationcited by 65
  11. 2020Federated Composite Optimizationcited by 49
  12. 2020Online Covariance Matrix Estimation in Stochastic Gradient Descentcited by 37
  13. 2019Communication-Efficient Accurate Statistical Estimationcited by 32
  14. 2019Normal Approximation for Stochastic Gradient Descent via Non-Asymptotic Rates of Martingale CLTcited by 30
  15. 2021The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communicationcited by 30
  16. 2021Federated f-Differential Privacycited by 29
  17. 2021Fast and Robust Online Inference with Stochastic Gradient Descent via Random Scalingcited by 28
  18. 2017Statistical inference using SGDcited by 21
  19. 2019Communication-Efficient Local Decentralized SGD Methodscited by 21
  20. 2018First-order Newton-type Estimator for Distributed Estimation and Inferencecited by 16
  21. 2017Statistical Inference for the Population Landscape via Moment Adjusted Stochastic Gradientscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 0; 2026: 1.

2 papers.

  1. 2022Fast Composite Optimization and Statistical Recovery in Federated Learningcited by 12
  2. 2026Online Inference for Quantile Temporal Difference Learning in Distributional Reinforcement Learningcited by 1