mlsanityData to 7 Oct 2026

First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints

arXiv 2603.05774 ↗First posted 6 Mar 2026cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper addresses the distributed stochastic minimax optimization problem subject to stochastic constraints. We propose a novel first-order Softmax-Weighted Switching Gradient method tailored for federated learning. Under full client participation, our algorithm achieves the standard tildeO(ε-4) oracle complexity to satisfy a unified bound ε for both the optimality gap and feasibility tolerance. We extend our theoretical analysis to the practical partial participation regime by quantifying client sampling noise through a stochastic superiority assumption. Furthermore, by relaxing standard boundedness assumptions on the objective functions, we establish a strictly tighter lower bound for the softmax hyperparameter. We provide a unified error decomposition and establish a sharp O(logfrac1δ) high-probability convergence guarantee. Ultimately, our framework demonstrates that a single-loop primal-only switching mechanism provides a stable alternative for optimizing worst-case client performance, effectively bypassing the hyperparameter sensitivity and convergence oscillations often encountered in traditional primal-dual or penalty-based approaches. We verify the efficacy of our algorithm via experiment on the Neyman-Pearson (NP) classification, fair classification, and federated safe reinforcement learning tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 66 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

38 of 66 references.

References · 38

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  5. 2020The Future of Digital Health with Federated Learningcited by 625
  6. 2014Convex Optimization: Algorithms and Complexitycited by 569
  7. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  8. 2019Agnostic Federated Learningcited by 451
  9. 2019Fair Resource Allocation in Federated Learningcited by 389
  10. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
  11. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  12. 2021Distributionally Robust Federated Averagingcited by 104
26 more
  1. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
  2. 2020Fairness-aware Agnostic Federated Learningcited by 87
  3. 2024FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Modelcited by 73
  4. 2022Federated Learning via Inexact ADMMcited by 44
  5. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 35
  6. 2024Relaxed Contrastive Learning for Federated Learningcited by 32
  7. 2019Quadratically Regularized Subgradient Methods for Weakly Convex Optimization with Weakly Convex Constraintscited by 25
  8. 2022FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneitycited by 25
  9. 2020Rate-improved Inexact Augmented Lagrangian Method for Constrained Nonconvex Optimizationcited by 21
  10. 2022First-Order Methods for Nonsmooth Nonconvex Functional Constrained Optimization with or without Slater Pointscited by 17
  11. 2023Resilient Constrained Learningcited by 15
  12. 2023Oracle Complexity of Single-Loop Switching Subgradient Methods for Non-Smooth Weakly Convex Functional Constrained Optimizationcited by 11
  13. 2025A single-loop SPIDER-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth optimizationcited by 8
  14. 2024Towards Diverse Device Heterogeneous Federated Learning via Task Arithmetic Knowledge Integrationcited by 5
  15. 2025Stochastic Smoothed Primal-Dual Algorithms for Nonconvex Optimization with Linear Inequality Constraintscited by 5
  16. 2025Safe-EF: Error Feedback for Nonsmooth Constrained Optimizationcited by 5
  17. 2019On Modification of an Adaptive Stochastic Mirror Descent Algorithm for Convex Optimization Problems with Functional Constraintscited by 4
  18. 2023Task-Robust Pre-Training for Worst-Case Downstream Adaptationcited by 3
  19. 2018Mirror Descent and Constrained Online Optimization Problemscited by 2
  20. 2019Mirror Descent for Constrained Optimization Problems with Large Subgradient Valuescited by 2
  21. 2024A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needscited by 2
  22. 2026FedSGM: A Unified Framework for Constraint Aware, Bidirectionally Compressed, Multi-Step Federated Optimizationcited by 2
  23. 2024Unveiling the Cycloid Trajectory of EM Iterations in Mixed Linear Regressioncited by 1
  24. 2024Optimal Primal-Dual Algorithm with Last iterate Convergence Guarantees for Stochastic Convex Optimization Problemscited by 1
  25. 2025Optimization via First-Order Switching Methods: Skew-Symmetric Dynamics and Optimistic Discretizationcited by 1
  26. 2025Structural Properties, Cycloid Trajectories and Non-Asymptotic Guarantees of EM Algorithm for Mixed Linear Regressioncited by 1

Built on by

Not cited yet.