A proximal subgradient method for nonconvex stochastic optimization under the Kurdyka-Lojasiewicz condition
Not on the map. Why?
Abstract (arXiv)
This work introduces a proximal stochastic subgradient method for minimizing the sum of an expected cost, whose integrand is potentially nonsmooth and nonconvex, and a lower semicontinuous, prox-bounded function. We target a broad class of integrands obeying a nonsmooth, localized variant of the descent lemma in the decision variable, a structural assumption that simultaneously covers smooth losses with Lipschitz gradient and differences of such losses with convex functions. At each iteration the expected cost is replaced by a sample average that is progressively refined, and the proximal-subgradient stepsize is selected by an Armijo-type line search enforcing a sufficient-decrease property up to stochastic errors induced by the sample-based approximation. This framework accommodates substantially more general problem formulations than existing methods, in particular, it requires neither (weak) convexity of the regularizer nor a uniform bound on the variance of the stochastic oracle, and our analysis yields convergence guarantees that are new even in the smooth setting. Specifically, we establish almost sure convergence of the sequence of function values and stationarity of every accumulation point of the trajectories under the relaxed requirement that the sample-size sequence be merely nondecreasing and unbounded, with no prescribed growth rate. Leveraging the Kurdyka-Lojasiewicz (KL) property, we further upgrade this subsequential guarantee to convergence of the whole trajectory to a single stationary point. Finally, for exponential-type KL desingularizing functions and polynomially growing sample sizes, we derive explicit polynomial convergence rates, up to a logarithmic factor, for both the function values and the iterates.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 89 references that are in the map. First posted on arXiv 5 Aug 2026.
Builds on
38 of 89 references.
References · 38
- 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
- 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
- 2018Stochastic model-based minimization of weakly convex functionscited by 157
- 2013Mini-batch Stochastic Approximation Methods for Nonconvex Stochastic Composite Optimizationcited by 142
- 2013Accelerated Gradient Methods for Nonconvex Nonlinear and Stochastic Programmingcited by 74
- 2013Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optimacited by 66
- 2017Stochastic Methods for Composite and Weakly Convex Optimization Problemscited by 48
- 2019ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimizationcited by 37
- 2016Calculus of the exponent of Kurdyka-Lojasiewicz inequality and its applications to linear convergence of first-order methodscited by 35
- 2017Extragradient method with variance reduction for stochastic variational inequalitiescited by 33
- 2017Proximally Guided Stochastic Subgradient Method for Nonsmooth, Nonconvex Problemscited by 30
- 2017Learning from Ambiguously Labeled Face Imagescited by 29
26 more
- 2019A Hybrid Stochastic Optimization Framework for Stochastic Composite Nonconvex Optimizationcited by 27
- 2014Splitting methods with variable metric for KL functionscited by 26
- 2022First-Order Methods for Nonsmooth Nonconvex Functional Constrained Optimization with or without Slater Pointscited by 17
- 2024Fundamental Convergence Analysis of Sharpness-Aware Minimizationcited by 14
- 2020Variable Smoothing for Weakly Convex Composite Functionscited by 12
- 2023Stochastic Optimization under Hidden Convexitycited by 9
- 2015Accelerating the DC algorithm for smooth functionscited by 8
- 2015On the Convergence of the Iterative Shrinkage/Thresholding Algorithm With a Weakly Convex Penaltycited by 6
- 2013A Method for Finding Structured Sparse Solutions to Non-negative Least Squares Problems with Applicationscited by 5
- 2022A Zeroth-order Proximal Stochastic Gradient Method for Weakly Convex Stochastic Optimizationcited by 5
- 2022Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimizationcited by 5
- 2019Stochastic Proximal Methods for Non-Smooth Non-Convex Constrained Sparse Optimizationcited by 4
- 2017Incremental constraint projection methods for monotone stochastic variational inequalitiescited by 3
- 2018Asynchronous Variance-reduced Block Schemes for Composite Nonconvex Stochastic Optimization: Block-specific Steplengths and Adapted Batch-sizescited by 3
- 2023Partial-Label Regressioncited by 3
- 2017On variance reduction for stochastic smooth convex optimization with multiplicative noisecited by 2
- 2020Stochastic Proximal Gradient Methods for Nonconvex Problems in Hilbert Spacescited by 2
- 2023The Boosted Double-Proximal Subgradient Algorithm for Nonconvex Optimizationcited by 2
- 2024Stochastic ISTA/FISTA Adaptive Step Search Algorithms for Convex Composite Optimizationcited by 2
- 2020Variance-Reduced Accelerated First-order Methods: Central Limit Theorems and Confidence Statementscited by 1
- 2023Nonasymptotic upper estimates for errors of the sample average approximation method to solve risk averse stochastic programscited by 1
- 2023Understanding the Douglas-Rachford splitting method through the lenses of Moreau-type envelopescited by 1
- 2025Randomized block proximal method with locally Lipschitz continuous gradientcited by 1
- 2025Nonmonotone subgradient methods based on a local descent lemmacited by 1
- 2025Shadow splitting methods for nonconvex optimisation: epi-approximation, convergence and saddle point avoidancecited by 1
- 2026A Nonmonotone Descent Method for Optimization Problems Defined by Upper-C2 Functions over Submanifoldscited by 1
Built on by
Not cited yet.