# Convergence Analysis of Proximal Gradient with Momentum for Nonconvex Optimization

arXiv: https://arxiv.org/abs/1705.04925 · HTML: https://mlsanity.com/paper/1705.04925/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2017.

## Built on by (most cited first, 18 of 18)

- [A Survey on Efficient Training of Transformers](https://mlsanity.com/paper/2302.01107/) (2023)
- [Proximal Gradient Descent-Ascent: Variable Convergence under KL Geometry](https://mlsanity.com/paper/2102.04653/) (2021)
- [Restarted Nonconvex Accelerated Gradient Descent: No More Polylogarithmic Factor in the O(ε-7/4) Complexity](https://mlsanity.com/paper/2201.11411/) (2022)
- [Unified Group Fairness on Federated Learning](https://mlsanity.com/paper/2111.04986/) (2021)
- [A Fast and Convergent Proximal Algorithm for Regularized Nonconvex and Nonsmooth Bi-level Optimization](https://mlsanity.com/paper/2203.16615/) (2022)
- [TAG: Task-based Accumulated Gradients for Lifelong learning](https://mlsanity.com/paper/2105.05155/) (2021)
- [Accelerated Proximal Alternating Gradient-Descent-Ascent for Nonconvex Minimax Machine Learning](https://mlsanity.com/paper/2112.11663/) (2021)
- [Bayesian Learning of Parameterised Quantum Circuits](https://mlsanity.com/paper/2206.07559/) (2022)
- [An Accelerated Block Proximal Framework with Adaptive Momentum for Nonconvex and Nonsmooth Optimization](https://mlsanity.com/paper/2308.12126/) (2023)
- [Structured Sparse Non-negative Matrix Factorization with L20-Norm for scRNA-seq Data Analysis](https://mlsanity.com/paper/2104.13171/) (2021)
- [When Stability meets Sufficiency: Informative Explanations that do not Overwhelm](https://mlsanity.com/paper/2109.06181/) (2021)
- [Graph Regularized NMF with L20-norm for Unsupervised Feature Learning](https://mlsanity.com/paper/2403.10910/) (2024)
- [Low-rank Tensor Learning with Nonconvex Overlapped Nuclear Norm Regularization](https://mlsanity.com/paper/2205.03059/) (2022)
- [Adaptive Lipschitz-Free Proximal Gradient Methods for Stochastic Composite Nonconvex Optimization](https://mlsanity.com/paper/2502.21099/) (2025)
- [Projective Proximal Gradient Descent for A Class of Nonconvex Nonsmooth Optimization Problems: Fast Convergence Without Kurdyka-Lojasiewicz (KL) Property](https://mlsanity.com/paper/2304.10499/) (2023)
- [Globally Convergent Accelerated Algorithms for Multilinear Sparse Logistic Regression with ℓ0-constraints](https://mlsanity.com/paper/2309.09239/) (2023)
- [Federated Online Learning for Heterogeneous Multisource Streaming Data](https://mlsanity.com/paper/2508.06652/) (2025)
- [An Inertial Block Proximal Linearized Method with Adaptive Momentum for Nonconvex and Nonsmooth Optimization](https://mlsanity.com/paper/2608.05502/) (2026)

## Abstract (arXiv)

In many modern machine learning applications, structures of underlying mathematical models often yield nonconvex optimization problems. Due to the intractability of nonconvexity, there is a rising need to develop efficient methods for solving general nonconvex problems with certain performance guarantee. In this work, we investigate the accelerated proximal gradient method for nonconvex programming (APGnc). The method compares between a usual proximal gradient step and a linear extrapolation step, and accepts the one that has a lower function value to achieve a monotonic decrease. In specific, under a general nonsmooth and nonconvex setting, we provide a rigorous argument to show that the limit points of the sequence generated by APGnc are critical points of the objective function. Then, by exploiting the Kurdyka-Lojasiewicz (KL) property for a broad class of functions, we establish the linear and sub-linear convergence rates of the function value sequence generated by APGnc. We further propose a stochastic variance reduced APGnc (SVRG-APGnc), and establish its linear convergence under a special case of the KL property. We also extend the analysis to the inexact version of these methods and develop an adaptive momentum strategy that improves the numerical performance.