mlsanityData to 7 Oct 2026

Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates

arXiv 1911.09030 ↗First posted 20 Nov 2019cs.LG cs.DC stat.MLcited by 26

Not on the map. Why?

Abstract (arXiv)

When scaling distributed training, the communication overhead is often the bottleneck. In this paper, we propose a novel SGD variant with reduced communication and adaptive learning rates. We prove the convergence of the proposed algorithm for smooth but non-convex problems. Empirical results show that the proposed algorithm significantly reduces the communication overhead, which, in turn, reduces the training time by up to 30% for the 1B word dataset.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 6; 2023: 7; 2024: 1; 2025: 3; 2026: 1.

26 papers.

Sort:

  1. 2021A Field Guide to Federated Optimizationcited by 312
  2. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  3. 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
  4. 2021FedNL: Making Newton-Type Methods Applicable to Federated Learningcited by 50
  5. 2021Local Adaptivity in Federated Learning: Convergence and Consistencycited by 31
  6. 2021Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Accelerationcited by 25
  7. 2021Toward Communication Efficient Adaptive Gradient Methodcited by 25
  8. 2022FedNew: A Communication-Efficient and Privacy-Preserving Newton-Type Method for Federated Learningcited by 23
  9. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  10. 2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedupcited by 23
  11. 2022FedShuffle: Recipes for Better Use of Local Work in Federated Learningcited by 19
  12. 2022Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Datacited by 9
14 more
  1. 2023Adaptive Federated Learning with Auto-Tuned Clientscited by 7
  2. 2023Locally Adaptive Federated Learningcited by 7
  3. 2025Efficient Distributed Optimization under Heavy-Tailed Noisecited by 7
  4. 2023Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentumcited by 3
  5. 2021Local AdaGrad-Type Algorithm for Stochastic Convex-Concave Optimizationcited by 2
  6. 2023FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Datacited by 2
  7. 2024Efficient Adaptive Federated Optimizationcited by 2
  8. 2025Gradient Correction in Federated Learning with Adaptive Optimizationcited by 1
  9. 2022Better Methods and Theory for Federated Learning: Compression, Client Selection and Heterogeneitycited by 0
  10. 2022On the Stability Analysis of Open Federated Learning Systemscited by 0
  11. 2023Preconditioned Federated Learningcited by 0
  12. 2023Data-driven prediction of tool wear using Bayesian-regularized artificial neural networkscited by 0
  13. 2025FedDuA: Doubly Adaptive Federated Learningcited by 0
  14. 2026LoDAdaC: a unified local training-based decentralized framework with adaptive gradients and compressed communicationcited by 0