mlsanityData to 7 Oct 2026

Stochastic Weakly Convex Optimization Beyond Lipschitz Continuity

arXiv 2401.13971 ↗First posted 25 Jan 2024math.OC cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper considers stochastic weakly convex optimization without the standard Lipschitz continuity assumption. Based on new adaptive regularization (stepsize) strategies, we show that a wide class of stochastic algorithms, including the stochastic subgradient method, preserve the O ( 1 / √K) convergence rate with constant failure rate. Our analyses rest on rather weak assumptions: the Lipschitz parameter can be either bounded by a general growth function of |x| or locally estimated through independent random samples.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 33 references that are in the map. First posted on arXiv 25 Jan 2024.

Builds on

28 of 33 references.

References · 28

  1. 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
  2. 2018Stochastic model-based minimization of weakly convex functionscited by 157
  3. 2018On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizescited by 155
  4. 2023Learning-Rate-Free Learning by D-Adaptationcited by 113
  5. 2020Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clippingcited by 104
  6. 2023DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedulecited by 85
  7. 2023Revisiting Gradient Clipping: Stochastic bias and tight convergence guaranteescited by 83
  8. 2018Stochastic (Approximate) Proximal Point Methods: Convergence, Optimality, and Adaptivitycited by 61
  9. 2016Efficiency of minimizing compositions of convex functions and smooth mapscited by 59
  10. 2023Convex and Non-convex Optimization Under Generalized Smoothnesscited by 44
  11. 2021Stability and Convergence of Stochastic Gradient Clipping: Beyond Lipschitz Continuity and Smoothnesscited by 37
  12. 2022Policy Gradient in Robust MDPs with Global Convergence Guaranteecited by 35
16 more
  1. 2018Subgradient methods for sharp weakly convex functionscited by 33
  2. 2020Convergence of a Stochastic Gradient Method with Momentum for Non-Smooth Non-Convex Optimizationcited by 32
  3. 2019Low-rank matrix recovery with composite optimization: good conditioning and rapid convergencecited by 31
  4. 2017Proximally Guided Stochastic Subgradient Method for Nonsmooth, Nonconvex Problemscited by 30
  5. 2017Solving (most) of a set of quadratic equalities: Composite optimization for robust phase retrievalcited by 24
  6. 2023Adaptive Proximal Gradient Method for Convex Optimizationcited by 21
  7. 2017Convergence Rates for Deterministic and Stochastic Subgradient Methods Without Lipschitz Continuitycited by 18
  8. 2018On the Convergence Rate of Stochastic Mirror Descent for Nonsmooth Nonconvex Optimizationcited by 18
  9. 2017"Relative-Continuity" for Non-Lipschitz Non-Smooth Convex Optimization using Stochastic (or Deterministic) Mirror Descentcited by 15
  10. 2023A Stochastic Proximal Polyak Step Sizecited by 15
  11. 2019Stochastic algorithms with geometric step decay converge linearly on sharp functionscited by 14
  12. 2021Minibatch and Momentum Model-based Methods for Stochastic Weakly Convex Optimizationcited by 10
  13. 2023Trust Region Methods For Nonconvex Stochastic Optimization Beyond Lipschitz Smoothnesscited by 8
  14. 2018Stochastic model-based minimization under high-order growthcited by 7
  15. 2023Revisiting Subgradient Method: Complexity and Convergence Beyond Lipschitz Continuitycited by 2
  16. 2023A Unified Analysis on the Subgradient Upper Bounds for the Subgradient Methods Minimizing Composite Nonconvex, Nonsmooth and Non-Lipschitz Functionscited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2024Federated Smoothing Proximal Gradient for Quantile Regression with Non-Convex Penaltiescited by 0
  2. 2025Stochastic Weakly Convex Optimization Under Heavy-Tailed Noisescited by 0
  3. 2025Quantitative Convergence Analysis of Projected Stochastic Gradient Descent for Non-Convex Losses via the Goldstein Subdifferentialcited by 0