mlsanityData to 7 Oct 2026

Adaptive Federated Learning with Auto-Tuned Clients

arXiv 2306.11201 ↗First posted 19 Jun 2023cs.LG cs.DC math.OCcited by 7

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a distributed machine learning framework where the global model of a central server is trained via multiple collaborative steps by participating clients without sharing their data. While being a flexible framework, where the distribution of local data, participation rate, and computing power of each client can greatly vary, such flexibility gives rise to many new challenges, especially in the hyperparameter tuning on the client side. We propose Delta-SGD, a simple step size rule for SGD that enables each client to use its own step size by adapting to the local smoothness of the function each client is optimizing. We provide theoretical and empirical results where the benefit of the client adaptivity is shown in various FL scenarios.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 50 of its 71 references that are in the map. First posted on arXiv 19 Jun 2023.

Builds on

50 of 71 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  7. 2021Convex optimizationcited by 2,479
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  9. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  10. 2020Adaptive Federated Optimizationcited by 1,036
  11. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  12. 2021Model-Contrastive Federated Learningcited by 762
38 more
  1. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  2. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  3. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  4. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  5. 2019Asynchronous Federated Optimizationcited by 264
  6. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  7. 2018AdaGrad stepsizes: Sharp convergence over nonconvex landscapescited by 182
  8. 2013On the Convergence of Decentralized Gradient Descentcited by 158
  9. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  10. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 152
  11. 2018Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptroncited by 143
  12. 2020Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergencecited by 137
  13. 2016Coin Betting and Parameter-Free Online Learningcited by 131
  14. 2019Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Ratescited by 119
  15. 2023Learning-Rate-Free Learning by D-Adaptationcited by 113
  16. 2017Optimal algorithms for smooth and strongly convex distributed optimization in networkscited by 81
  17. 2019Variance Reduced Local SGD with Lower Communication Complexitycited by 80
  18. 2019Adaptive Gradient Descent without Descentcited by 79
  19. 2020Variance-Reduced Methods for Machine Learningcited by 79
  20. 2019Revisiting the Polyak step sizecited by 55
  21. 2013Fast Convergence of Stochastic Gradient Descent under a Strong Growth Conditioncited by 54
  22. 2018Online Adaptive Methods, Universality and Accelerationcited by 51
  23. 2017Communication-Efficient Algorithms for Decentralized and Stochastic Optimizationcited by 42
  24. 2020Advances in Asynchronous Parallel and Distributed Optimizationcited by 40
  25. 2018Stochastic Quasi-Gradient Methods: Variance Reduction via Jacobian Sketchingcited by 34
  26. 2021Local Adaptivity in Federated Learning: Convergence and Consistencycited by 31
  27. 2018A Dual Approach for Optimal Algorithms in Distributed Optimization over Networkscited by 27
  28. 2020On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settingscited by 27
  29. 2019Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Ratescited by 26
  30. 2017Accelerated Distributed Nesterov Gradient Descentcited by 24
  31. 2023Adaptive proximal algorithms for convex optimization under local Lipschitz continuity of the gradientcited by 19
  32. 2023Automatic Gradient Descent: Deep Learning without Hyperparameterscited by 14
  33. 2022Grad-GradaGrad? A Non-Monotone Adaptive Stochastic Gradient Methodcited by 9
  34. 2023Green Federated Learningcited by 8
  35. 2021Convergence and Stability of the Stochastic Proximal Point Algorithm with Momentumcited by 7
  36. 2017On the linear convergence of the stochastic gradient method with constant step-sizecited by 6
  37. 2022PersA-FL: Personalized Asynchronous Federated Learningcited by 5
  38. 2022Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomographycited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 4.

7 papers.

  1. 2023Locally Adaptive Federated Learningcited by 7
  2. 2024Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methodscited by 4
  3. 2025FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learningcited by 4
  4. 2024Federated Learning: A Stochastic Approximation Approachcited by 0
  5. 2025Communication Efficient Federated Learning with Linear Convergence on Heterogeneous Datacited by 0
  6. 2025Adaptive Federated Learning via Dynamical System Modelcited by 0
  7. 2025Why Federated Optimization Fails to Achieve Perfect Fitting? A Theoretical Perspective on Client-Side Optimacited by 0