mlsanityData to 7 Oct 2026

Stabilized Proximal-Point Methods for Federated Optimization

arXiv 2407.07084 ↗First posted 9 Jul 2024cs.LG math.OCcited by 6

Not on the map. Why?

Abstract (arXiv)

In developing efficient optimization algorithms, it is crucial to account for communication constraints -- a significant challenge in modern Federated Learning. The best-known communication complexity among non-accelerated algorithms is achieved by DANE, a distributed proximal-point algorithm that solves local subproblems at each iteration and that can exploit second-order similarity among individual functions. However, to achieve such communication efficiency, the algorithm requires solving local subproblems sufficiently accurately resulting in slightly sub-optimal local complexity. Inspired by the hybrid-projection proximal-point method, in this work, we propose a novel distributed algorithm S-DANE. Compared to DANE, this method uses an auxiliary sequence of prox-centers while maintaining the same deterministic communication complexity. Moreover, the accuracy condition for solving the subproblem is milder, leading to enhanced local computation efficiency. Furthermore, S-DANE supports partial client participation and arbitrary stochastic local solvers, making it attractive in practice. We further accelerate S-DANE and show that the resulting algorithm achieves the best-known communication complexity among all existing methods for distributed convex optimization while still enjoying good local computation efficiency as S-DANE. Finally, we propose adaptive variants of both methods using line search, obtaining the first provably efficient adaptive algorithms that could exploit local second-order similarity without the prior knowledge of any parameters.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 38 of its 63 references that are in the map. First posted on arXiv 9 Jul 2024.

Builds on

38 of 63 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  7. 2014Convex Optimization: Algorithms and Complexitycited by 569
  8. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  9. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  10. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 401
  11. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  12. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
26 more
  1. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  2. 2013Communication Efficient Distributed Optimization using an Approximate Newton-type Methodcited by 131
  3. 2020Federated Accelerated Stochastic Gradient Descentcited by 85
  4. 2019First Analysis of Local GD on Heterogeneous Datacited by 80
  5. 2015A Universal Catalyst for First-Order Optimizationcited by 54
  6. 2021Acceleration Methodscited by 48
  7. 2016AIDE: Fast and Communication Efficient Distributed Optimizationcited by 42
  8. 2020Statistically Preconditioned Accelerated Gradient Method for Distributed Optimizationcited by 29
  9. 2022RandProx: Primal-Dual Optimization Algorithms with Randomized Proximal Updatescited by 27
  10. 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!cited by 22
  11. 2022AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimationcited by 21
  12. 2022Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Proxcited by 19
  13. 2022Faster federated optimization under second-order similaritycited by 18
  14. 2021Acceleration in Distributed Optimization under Similaritycited by 17
  15. 2014Gradient Sliding for Composite Optimizationcited by 14
  16. 2022Optimization with Access to Auxiliary Informationcited by 12
  17. 2023Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysiscited by 11
  18. 2024Federated Optimization with Doubly Regularized Drift Correctioncited by 10
  19. 2024Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reductioncited by 9
  20. 2022RECAPP: Crafting a More Efficient Catalyst for Convex Optimizationcited by 8
  21. 2016Accelerated gradient sliding for structured convex optimizationcited by 7
  22. 2023Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalitiescited by 7
  23. 2019Adaptive Catalyst for Smooth Convex Optimizationcited by 6
  24. 2016Generalizing the optimized gradient method for smooth convex minimizationcited by 5
  25. 2019Contracting Proximal Methods for Smooth Convex Optimizationcited by 4
  26. 2024SPAM: Stochastic Proximal Point Method with Momentum Variance Reduction for Non-convex Cross-Device Federated Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025FedMuon: Federated Learning with Bias-corrected LMO-based Optimizationcited by 7
  2. 2025Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participationcited by 3
  3. 2025Exploiting Similarity for Computation and Communication-Efficient Decentralized Optimizationcited by 1
  4. 2025Non-Convex Federated Optimization under Cost-Aware Client Selectioncited by 1
  5. 2025Why Federated Optimization Fails to Achieve Perfect Fitting? A Theoretical Perspective on Client-Side Optimacited by 0
  6. 2026Accelerated Methods with Complexity Separation Under Data Similarity for Federated Learning Problemscited by 0