mlsanityData to 13 Oct 2026

Katyusha X: Practical Momentum Method for Stochastic Sum-of-Nonconvex Optimization

arXiv 1802.03866 ↗First posted 12 Feb 2018cs.LG cs.DS math.OCcited by 15

Not on the map. Why?

Abstract (arXiv)

The problem of minimizing sum-of-nonconvex functions (i.e., convex functions that are average of non-convex ones) is becoming increasingly important in machine learning, and is the core machinery for PCA, SVD, regularized Newton's method, accelerated non-convex optimization, and more. We show how to provably obtain an accelerated stochastic algorithm for minimizing sum-of-nonconvex functions, by textitadding one additional line to the well-known SVRG method. This line corresponds to momentum, and shows how to directly apply momentum to the finite-sum stochastic minimization of sum-of-nonconvex functions. As a side result, our method enjoys linear parallel speed-up using mini-batch.

Key facts. Cited by 15 papers in the map (data through 13 Oct 2026). First posted on arXiv 12 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 1; 2022: 3; 2023: 4; 2024: 5; 2025: 1; 2026: 1.

15 papers.

Sort:

  1. 2022Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimizationcited by 15
  2. 2022Distributionally Robust Optimization via Ball Oracle Accelerationcited by 14
  3. 2023Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysiscited by 11
  4. 2023Faster Stochastic Algorithms for Minimax Optimization under Polyak--Lojasiewicz Conditionscited by 5
  5. 2022Adaptive Accelerated (Extra-)Gradient Methods with Variance Reductioncited by 4
  6. 2024Second-order Information Promotes Mini-Batch Robustness in Variance-Reduced Gradientscited by 3
  7. 2021Delayed Projection Techniques for Linearly Constrained Problems: Convergence Rates, Acceleration, and Applicationscited by 2
  8. 2024Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reductioncited by 2
  9. 2024Accelerated Methods with Compressed Communications for Distributed Optimization Problems under Data Similaritycited by 2
  10. 2023Sharpened Lazy Incremental Quasi-Newton Methodcited by 1
  11. 2024Decentralized Sum-of-Nonconvex Optimizationcited by 1
  12. 2025Non-Convex Federated Optimization under Cost-Aware Client Selectioncited by 1
3 more
  1. 2023Accelerated Gradient Algorithms with Adaptive Subspace Search for Instance-Faster Optimizationcited by 0
  2. 2024Efficient Continual Finite-Sum Minimizationcited by 0
  3. 2026Accelerated Methods with Complexity Separation Under Data Similarity for Federated Learning Problemscited by 0