mlsanityData to 7 Oct 2026

MoSSP: A Momentum-Based Single-Loop Stochastic Penalty Method for Nonconvex Constrained DC-Regularized Optimization

arXiv 2605.29635 ↗First posted 28 May 2026math.OC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we study a structured class of nonconvex constrained stochastic problems with difference-of-convex (DC) regularization, where the feasible set is possibly nonconvex and the concave part of the DC regularizer is allowed to be nonsmooth. The fundamental challenge lies in maintaining feasibility for nonconvex constraints while achieving favorable oracle complexity. Although single-loop algorithms efficiently solve unconstrained DC optimization problems, their potential for constrained optimization with DC structure remains largely unexplored. To address this gap, we develop MoSSP, a Momentum-based Single-loop Stochastic Penalty method for such problems with provable complexity guarantees. The key idea is to apply a single stochastic proximal-gradient step to the Moreau envelope of the penalty plus the convex DC part, with the concave part's proximal mapping computed in parallel. We derive two algorithm variants: a Polyak-momentum version with O(varepsilon-4) oracle complexity for finding stochastic varepsilon-KKT points, and an improved O(varepsilon-3) version incorporating recursive momentum. Experimental results demonstrate the effectiveness of the proposed algorithms.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 46 of its 74 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

46 of 74 references.

References · 46

  1. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  2. 2019Lower Bounds for Non-Convex Stochastic Optimizationcited by 303
  3. 2019Momentum-Based Variance Reduction in Non-Convex SGDcited by 302
  4. 2018SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimatorcited by 283
  5. 2017SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradientcited by 218
  6. 2017Positive-Unlabeled Learning with Non-Negative Risk Estimatorcited by 217
  7. 2018Stochastic model-based minimization of weakly convex functionscited by 157
  8. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 144
  9. 2020An Improved Analysis of Stochastic Gradient Descent with Momentumcited by 117
  10. 2022Towards understanding how momentum improves generalization in deep learningcited by 40
  11. 2020Sequential Quadratic Optimization for Nonlinear Equality Constrained Stochastic Optimizationcited by 30
  12. 2013A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problemscited by 25
34 more
  1. 2021An Adaptive Stochastic Sequential Quadratic Programming with Differentiable Exact Augmented Lagrangianscited by 25
  2. 2021Worst-Case Complexity of an SQP Method for Nonlinear Equality Constrained Stochastic Optimizationcited by 19
  3. 2019Complexity of Proximal augmented Lagrangian for nonconvex optimization with nonlinear equality constraintscited by 17
  4. 2018Stochastic Optimization for DC Functions and Non-smooth Non-convex Regularizers with Non-asymptotic Convergencecited by 16
  5. 2021Inequality Constrained Stochastic Nonlinear Optimization via Active-Set Sequential Quadratic Programmingcited by 16
  6. 2023Complexity of Single Loop Algorithms for Nonlinear Programming with Stochastic Objective and Constraintscited by 16
  7. 2018ECC: Platform-Independent Energy-Constrained Deep Neural Network Compression via a Bilinear Regression Modelcited by 15
  8. 2022Large-scale Optimization of Partial AUC in a Range of False Positive Ratescited by 15
  9. 2020Momentum-based variance-reduced proximal stochastic gradient method for composite nonconvex stochastic optimizationcited by 14
  10. 2015Computing B-Stationary Points of Nonsmooth DC Programscited by 12
  11. 2016A proximal difference-of-convex algorithm with extrapolationcited by 12
  12. 2018Primal-dual stochastic gradient method for convex programs with many functional constraintscited by 12
  13. 2022An Adaptive Sampling Sequential Quadratic Programming Method for Equality Constrained Stochastic Optimizationcited by 11
  14. 2024Variance-reduced first-order methods for deterministically constrained stochastic nonconvex optimization with strong convergence guaranteescited by 11
  15. 2022Statistical Inference of Constrained Stochastic Optimization via Sketched Sequential Quadratic Programmingcited by 10
  16. 2013Penalty Methods with Stochastic Approximation for Stochastic Nonlinear Programmingcited by 9
  17. 2024Non-convex Stochastic Composite Optimization with Polyak Momentumcited by 9
  18. 2025A single-loop SPIDER-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth optimizationcited by 8
  19. 2020Convergence rate analysis of a sequential convex programming method with line search for a class of constrained difference-of-convex optimization problemscited by 5
  20. 2023Constrained Optimization via Exact Augmented Lagrangian and Randomized Iterative Sketchingcited by 5
  21. 2023Almost-sure convergence of iterates and multipliers in stochastic sequential quadratic optimizationcited by 5
  22. 2025Single-loop Algorithms for Stochastic Non-convex Optimization with Weakly-Convex Constraintscited by 5
  23. 2021The difference of convex algorithm on Hadamard manifoldscited by 4
  24. 2024Single-Loop Stochastic Algorithms for Difference of Max-Structured Weakly Convex Functionscited by 4
  25. 2021Penalty and Augmented Lagrangian Methods for Constrained DC Programmingcited by 3
  26. 2025An Inexact Proximal Framework for Nonsmooth Riemannian Difference-of-Convex Optimizationcited by 2
  27. 2025Stochastic Difference-of-Convex Optimization with Momentumcited by 2
  28. 2025A nonmonotone extrapolated proximal gradient-subgradient algorithm beyond global Lipschitz gradient continuitycited by 2
  29. 2024An Inexact Bregman Proximal Difference-of-Convex Algorithm with Two Types of Relative Stopping Criteriacited by 1
  30. 2025Convergence Analysis of an Inexact MBA Method for Constrained DC Problemscited by 1
  31. 2025A smoothing moving balls approximation method for a class of conic-constrained difference-of-convex optimization problemscited by 1
  32. 2025An adaptive proximal safeguarded augmented Lagrangian method for nonsmooth DC problems with convex constraintscited by 1
  33. 2025Exchange Policy Optimization Algorithm for Semi-Infinite Safe Reinforcement Learningcited by 1
  34. 2026Augmented Lagrangian Multiplier Network for State-wise Safety in Reinforcement Learningcited by 1

Built on by

Not cited yet.