mlsanityData to 14 Oct 2026

Stochastic Methods for Composite and Weakly Convex Optimization Problems

arXiv 1703.08570 ↗First posted 24 Mar 2017math.OC math.ST stat.THcited by 48

Not on the map. Why?

Abstract (arXiv)

We consider minimization of stochastic functionals that are compositions of a (potentially) non-smooth convex function h and smooth function c and, more generally, stochastic weakly-convex functionals. We develop a family of stochastic methods---including a stochastic prox-linear algorithm and a stochastic (generalized) sub-gradient procedure---and prove that, under mild technical conditions, each converges to first-order stationary points of the stochastic objective. We provide experiments further investigating our methods on non-smooth phase retrieval problems; the experiments indicate the practical effectiveness of the procedures.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 9; 2023: 10; 2024: 6; 2025: 8; 2026: 4.

48 papers.

Sort:

  1. 2022Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Modelscited by 84
  2. 2022Gradient-Free Methods for Deterministic and Stochastic Nonsmooth Nonconvex Optimizationcited by 43
  3. 2021Oracle Complexity in Nonsmooth Nonconvex Optimizationcited by 39
  4. 2021Stability and Convergence of Stochastic Gradient Clipping: Beyond Lipschitz Continuity and Smoothnesscited by 37
  5. 2022Convergence of gradient descent for deep neural networkscited by 24
  6. 2025Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimizationcited by 21
  7. 2021Accelerated, Optimal, and Parallel: Some Results on Model-Based Stochastic Optimizationcited by 14
  8. 2021When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpointcited by 12
  9. 2022A Unified Convergence Theorem for Stochastic Optimization Methodscited by 12
  10. 2021Minibatch and Momentum Model-based Methods for Stochastic Weakly Convex Optimizationcited by 10
  11. 2023Non-Smooth Weakly-Convex Finite-sum Coupled Compositional Optimizationcited by 10
  12. 2021Stochastic Subgradient Descent Escapes Active Strict Saddles on Weakly Convex Functionscited by 9
36 more
  1. 2022A Projection-free Algorithm for Constrained Stochastic Multi-level Composition Optimizationcited by 8
  2. 2023A Normal Map-Based Proximal Stochastic Gradient Method: Convergence and Identification Propertiescited by 5
  3. 2023A Model-Based Method for Minimizing CVaR and Beyondcited by 5
  4. 2023Adam-family Methods with Decoupled Weight Decay in Deep Learningcited by 5
  5. 2026Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variancecited by 5
  6. 2021First-Order Methods for Convex Optimizationcited by 4
  7. 2021On the Self-Penalization Phenomenon in Feature Selectioncited by 4
  8. 2023Sharper Analysis for Minibatch Stochastic Proximal Point Methods: Stability, Smoothness, and Deviationcited by 4
  9. 2023A New Random Reshuffling Method for Nonsmooth Nonconvex Finite-sum Optimizationcited by 4
  10. 2024A Functional Model Method for Nonconvex Nonsmooth Conditional Stochastic Optimizationcited by 4
  11. 2021Second-order step-size tuning of SGD for non-convex optimizationcited by 3
  12. 2023Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networkscited by 3
  13. 2023Stochastic Subgradient Methods with Guaranteed Global Stability in Nonsmooth Nonconvex Optimizationcited by 3
  14. 2024On the Uniform Convergence of Subdifferentials in Stochastic Optimization and Learningcited by 3
  15. 2025On exploration of an interior mirror descent flow for stochastic nonconvex constrained problemcited by 3
  16. 2025Deep Learning as the Disciplined Construction of Tame Objectscited by 3
  17. 2022Efficient algorithms for implementing incremental proximal-point methodscited by 2
  18. 2024Developing Lagrangian-based Methods for Nonsmooth Nonconvex Optimizationcited by 2
  19. 2024Memory-Efficient 4-bit Preconditioned Stochastic Optimizationcited by 2
  20. 2026Musec: MomentUm SpEctral Clipping for Stable Muon-type Trainingcited by 2
  21. 2021OpReg-Boost: Learning to Accelerate Online Algorithms with Operator Regressioncited by 1
  22. 2022Randomized Coordinate Subgradient Method for Nonsmooth Composite Optimizationcited by 1
  23. 2023A New Inexact Proximal Linear Algorithm with Adaptive Stopping Criteria for Robust Phase Retrievalcited by 1
  24. 2024Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex functionscited by 1
  25. 2025The late-stage training dynamics of (stochastic) subgradient descent on homogeneous neural networkscited by 1
  26. 2021Robust Regression via Model Based Methodscited by 0
  27. 2022Consistent Approximations in Composite Optimizationcited by 0
  28. 2022Convergence of ease-controlled Random Reshuffling gradient Algorithms under Lipschitz smoothnesscited by 0
  29. 2023Dual Gauss-Newton Directions for Deep Learningcited by 0
  30. 2024On the Hardness of Meaningful Local Guarantees in Nonsmooth Nonconvex Optimizationcited by 0
  31. 2025Learning-Based Surrogate Method for Stochastic Optimization under Decision-Dependent Uncertainty with Adaptive Random Designscited by 0
  32. 2025Quantitative Convergence Analysis of Projected Stochastic Gradient Descent for Non-Convex Losses via the Goldstein Subdifferentialcited by 0
  33. 2025DualHash: A Stochastic Primal-Dual Algorithm with Theoretical Guarantee for Deep Hashingcited by 0
  34. 2025Nonconvex Penalized LAD Estimation in Partial Linear Models with DNNs: Asymptotic Analysis and Proximal Algorithmscited by 0
  35. 2026Decentralized Stochastic Subgradient-type Methods with Communication Compression for Nonsmooth Nonconvex Optimizationcited by 0
  36. 2026A proximal subgradient method for nonconvex stochastic optimization under the Kurdyka-Lojasiewicz conditioncited by 0