mlsanityData to 7 Oct 2026

Federated Composite Saddle Point Optimization

arXiv 2305.15643 ↗First posted 25 May 2023cs.LG math.OC stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) approaches for saddle point problems (SPP) have recently gained in popularity due to the critical role they play in machine learning (ML). Existing works mostly target smooth unconstrained objectives in Euclidean space, whereas ML problems often involve constraints or non-smooth regularization, which results in a need for composite optimization. Addressing these issues, we propose Federated Dual Extrapolation (FeDualEx), an extra-step primal-dual algorithm, which is the first of its kind that encompasses both saddle point optimization and composite objectives under the FL paradigm. Both the convergence analysis and the empirical evaluation demonstrate the effectiveness of FeDualEx in these challenging settings. In addition, even for the sequential version of FeDualEx, we provide rates for the stochastic composite saddle point setting which, to our knowledge, are not found in prior literature.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 55 of its 106 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

55 of 106 references.

References · 55

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2021Convex optimizationcited by 2,479
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  8. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  9. 2014Convex Optimization: Algorithms and Complexitycited by 569
  10. 2021A Field Guide to Federated Optimizationcited by 312
  11. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  12. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
38 more
  1. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  2. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  3. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  4. 2018Optimistic mirror descent in saddle-point problems: Going the extra (gradient) milecited by 132
  5. 2020Near-Optimal Algorithms for Minimax Optimizationcited by 119
  6. 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 106
  7. 2017On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimizationcited by 91
  8. 2020Federated Accelerated Stochastic Gradient Descentcited by 85
  9. 2018Lower complexity bounds of first-order methods for convex-concave bilinear saddle-point problemscited by 75
  10. 2021Fast Extra Gradient Methods for Smooth Structured Nonconvex-Nonconcave Minimax Problemscited by 61
  11. 2021FedNL: Making Newton-Type Methods Applicable to Federated Learningcited by 50
  12. 2020Federated Composite Optimizationcited by 49
  13. 2018Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimizationcited by 40
  14. 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
  15. 2022Federated Minimax Optimization: Improved Convergence Analyses and Algorithmscited by 37
  16. 2019Revisiting Stochastic Extragradientcited by 35
  17. 2021Optimistic Dual Extrapolation for Coherent Non-monotone Variational Inequalitiescited by 29
  18. 2021A Review of Adversarial Attack and Defense for Classification Methodscited by 29
  19. 2021Decentralized Local Stochastic Extra-Gradient for Variational Inequalitiescited by 25
  20. 2018Faster Rates for Convex-Concave Gamescited by 23
  21. 2021Efficient Algorithms for Federated Saddle Point Optimizationcited by 20
  22. 2014Global convergence of splitting methods for nonconvex composite optimizationcited by 19
  23. 2021Lower Bounds and Optimal Algorithms for Smooth and Strongly Convex Decentralized Optimization Over Time-Varying Networkscited by 19
  24. 2020Relative Lipschitzness in Extragradient Methods and a Direct Recipe for Accelerationcited by 18
  25. 2019A Decentralized Proximal Point-type Method for Saddle Point Problemscited by 17
  26. 2020Higher-order methods for convex-concave min-max optimization and monotone variational inequalitiescited by 16
  27. 2020Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysiscited by 15
  28. 2022Generalized Optimistic Methods for Convex-Concave Saddle Point Problemscited by 15
  29. 2019Sparse-GAN: Sparsity-constrained Generative Adversarial Network for Anomaly Detection in Retinal OCT Imagecited by 14
  30. 2020Distributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithmscited by 14
  31. 2021Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutionscited by 14
  32. 2021ADOM: Accelerated Decentralized Optimization Method for Time-Varying Networkscited by 13
  33. 2021On Accelerated Methods for Saddle-Point Problems with Composite Structurecited by 13
  34. 2020Federated Nonconvex Sparse Learningcited by 10
  35. 2021FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptationcited by 9
  36. 2019Sparse Generative Adversarial Networkcited by 8
  37. 2013Mirror Prox Algorithm for Multi-Term Composite Minimization and Semi-Separable Problemscited by 7
  38. 2017Stochastic Composite Least-Squares Regression with convergence rate O(1/n)cited by 7

50 most cited of 55 All 55 (JSON)

Built on by

Not cited yet.