mlsanityData to 7 Oct 2026

First Analysis of Local GD on Heterogeneous Data

arXiv 1909.04715 ↗First posted 10 Sep 2019cs.LG cs.DC cs.NAcited by 80

Not on the map. Why?

Abstract (arXiv)

We provide the first convergence analysis of local gradient descent for minimizing the average of smooth and convex but otherwise arbitrary functions. Problems of this form and local gradient descent as a solution method are of importance in federated learning, where each function is based on private data stored by a user on a mobile device, and the data of different users can be arbitrarily heterogeneous. We show that in a low accuracy regime, the method has the same communication complexity as gradient descent.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 22; 2023: 18; 2024: 11; 2025: 4; 2026: 2.

80 papers.

Sort:

  1. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  2. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  3. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 122
  4. 2021Decentralized Federated Averagingcited by 114
  5. 2021On Large-Cohort Training for Federated Learningcited by 73
  6. 2021Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacycited by 66
  7. 2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
  8. 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
  9. 2021Anarchic Federated Learningcited by 42
  10. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 35
  11. 2022Multi-Level Branched Regularization for Federated Learningcited by 34
  12. 2021Proximal and Federated Random Reshufflingcited by 33
38 more
  1. 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
  2. 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
  3. 2024Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learningcited by 29
  4. 2021Cost-Effective Federated Learning in Mobile Edge Networkscited by 25
  5. 2022Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerabilitycited by 25
  6. 2023Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneitycited by 25
  7. 2023Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneitycited by 25
  8. 2022Communication-Efficient Federated Learning with Accelerated Client Gradientcited by 24
  9. 2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedupcited by 23
  10. 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!cited by 22
  11. 2022On the Performance of Gradient Tracking with Local Updatescited by 21
  12. 2021Exploiting Heterogeneity in Robust Federated Best-Arm Identificationcited by 20
  13. 2022FedShuffle: Recipes for Better Use of Local Work in Federated Learningcited by 19
  14. 2022AsyncFedED: Asynchronous Federated Learning with Euclidean Distance based Adaptive Weight Aggregationcited by 19
  15. 2022Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Proxcited by 19
  16. 2023D2NO: Efficient Handling of Heterogeneous Input Function Spaces with Distributed Deep Neural Operatorscited by 18
  17. 2022GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexitycited by 17
  18. 2021Differential Privacy Meets Federated Learning under Communication Constraintscited by 15
  19. 2021On Convergence of Federated Averaging Langevin Dynamicscited by 14
  20. 2023Federated TD Learning over Finite-Rate Erasure Channels: Linear Speedup under Markovian Samplingcited by 14
  21. 2023SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learningcited by 14
  22. 2023Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiationcited by 13
  23. 2024Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptationcited by 12
  24. 2021On the Convergence of Shallow Neural Network Training with Randomly Masked Neuronscited by 11
  25. 2022Federated Random Reshuffling with Compression and Variance Reductioncited by 11
  26. 2021FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximationcited by 10
  27. 2023Over-the-Air Federated Learning and Optimizationcited by 10
  28. 2024Federated Optimization with Doubly Regularized Drift Correctioncited by 10
  29. 2021Handling Data Heterogeneity with Generative Replay in Collaborative Learning for Medical Imagingcited by 9
  30. 2022Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Datacited by 9
  31. 2021Iterated Vector Fields and Conservatism, with Applications to Federated Learningcited by 8
  32. 2023Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneitycited by 8
  33. 2023Improving Accelerated Federated Learning with Compression and Importance Samplingcited by 7
  34. 2021A Stochastic Newton Algorithm for Distributed Convex Optimizationcited by 6
  35. 2021Federated Semi-Supervised Learning with Class Distribution Mismatchcited by 6
  36. 2022Speeding up Heterogeneous Federated Learning with Sequentially Trained Superclientscited by 6
  37. 2022Personalized Federated Learning with Communication Compressioncited by 6
  38. 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6

50 most cited of 80 All 80 (JSON)