# A Momentum Accelerated Adaptive Cubic Regularization Method for Nonconvex Optimization

arXiv: https://arxiv.org/abs/2210.05987 · HTML: https://mlsanity.com/paper/2210.05987/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 21 references that are in the map.
- First posted on arXiv 12 Oct 2022.

## Builds on (most cited first, 11 of 11)

- [Matrix Completion has No Spurious Local Minimum](https://mlsanity.com/paper/1605.07272/) (2016)
- [A Geometric Analysis of Phase Retrieval](https://mlsanity.com/paper/1602.06664/) (2016)
- [Sub-sampled Cubic Regularization for Non-convex Optimization](https://mlsanity.com/paper/1705.05933/) (2017)
- [Gradient Descent Finds the Cubic-Regularized Non-Convex Newton Step](https://mlsanity.com/paper/1612.00547/) (2016)
- [Stochastic Variance-Reduced Cubic Regularized Newton Method](https://mlsanity.com/paper/1802.04796/) (2018)
- [Analysis of Krylov Subspace Solutions of Regularized Nonconvex Quadratic Problems](https://mlsanity.com/paper/1806.09222/) (2018)
- [Convergence of Cubic Regularization for Nonconvex Optimization under KL Property](https://mlsanity.com/paper/1808.07382/) (2018)
- [Cubic Regularization with Momentum for Nonconvex Optimization](https://mlsanity.com/paper/1810.03763/) (2018)
- [On the Quadratic Convergence of the Cubic Regularization Method under a Local Error Bound Condition](https://mlsanity.com/paper/1801.09387/) (2018)
- [An accelerated first-order method with complexity analysis for solving cubic regularization subproblems](https://mlsanity.com/paper/1911.12545/) (2019)
- [Approximate Secular Equations for the Cubic Regularization Subproblem](https://mlsanity.com/paper/2209.13268/) (2022)

## Built on by (most cited first, 2 of 2)

- [Improving Stochastic Cubic Newton with Momentum](https://mlsanity.com/paper/2410.19644/) (2024)
- [Parallel Trust-Region Approaches in Neural Network Training: Beyond Traditional Methods](https://mlsanity.com/paper/2312.13677/) (2023)

## Abstract (arXiv)

The cubic regularization method (CR) and its adaptive version (ARC) are popular Newton-type methods in solving unconstrained non-convex optimization problems, due to its global convergence to local minima under mild conditions. The main aim of this paper is to develop a momentum-accelerated adaptive cubic regularization method (ARCm) to improve the convergent performance. With the proper choice of momentum step size, we show the global convergence of ARCm and the local convergence can also be guaranteed under the KL property. Such global and local convergence can also be established when inexact solvers with low computational costs are employed in the iteration procedure. Numerical results for non-convex logistic regression and robust linear regression models are reported to demonstrate that the proposed ARCm significantly outperforms state-of-the-art cubic regularization methods (e.g., CR, momentum-based CR, ARC) and the trust region method. In particular, the number of iterations required by ARCm is less than 10% to 50% required by the most competitive method (ARC) in the experiments.