mlsanityData to 14 Oct 2026

Global convergence of the Heavy-ball method for convex optimization

arXiv 1412.7457 ↗First posted 23 Dec 2014math.OCcited by 61

Not on the map. Why?

Abstract (arXiv)

This paper establishes global convergence and provides global bounds of the convergence rate of the Heavy-ball method for convex optimization problems. When the objective function has Lipschitz-continuous gradient, we show that the Cesaro average of the iterates converges to the optimum at a rate of O(1/k) where k is the number of iterations. When the objective function is also strongly convex, we prove that the Heavy-ball iterates converge linearly to the unique optimum.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Dec 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 13; 2023: 6; 2024: 11; 2025: 8; 2026: 6.

61 papers.

Sort:

By area:

  1. 2024The AdEMAMix Optimizer: Better, Faster, Oldercited by 54
  2. 2025Implicit Bias of Spectral Descent and Muon on Multiclass Separable Datacited by 41
  3. 2022On Almost Sure Convergence Rates of Stochastic Gradient Methodscited by 35
  4. 2021Fast Margin Maximization via Dual Accelerationcited by 33
  5. 2023Momentum Provably Improves Error Feedback!cited by 31
  6. 2022Provable Adaptivity of Adam under Non-uniform Smoothnesscited by 30
  7. 2022Automatic differentiation of nonsmooth iterative algorithmscited by 25
  8. 2021Does Momentum Change the Implicit Regularization on Separable Data?cited by 22
  9. 2024On Convergence of Adam for Stochastic Optimization under Relaxed Assumptionscited by 21
  10. 2022Restarted Nonconvex Accelerated Gradient Descent: No More Polylogarithmic Factor in the O(ε-7/4) Complexitycited by 17
  11. 2024Revisiting Convergence of AdaGrad with Relaxed Assumptionscited by 15
  12. 2022On the fast convergence of minibatch heavy ball momentumcited by 13
38 more
  1. 2024Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performancecited by 13
  2. 2022Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-Lojasiewicz Functions when the Non-Convexity is Averaged-Outcited by 11
  3. 2021Minibatch and Momentum Model-based Methods for Stochastic Weakly Convex Optimizationcited by 10
  4. 2024Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networkscited by 9
  5. 2021Escaping Saddle Points Faster with Stochastic Momentumcited by 8
  6. 2021Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimizationcited by 8
  7. 2023Exponential convergence rates for momentum stochastic gradient descent in the overparametrized settingcited by 8
  8. 2023High Probability Convergence of Adam Under Unbounded Gradients and Affine Variance Noisecited by 8
  9. 2021The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methodscited by 7
  10. 2021Heavy Ball Momentum for Conditional Gradientcited by 7
  11. 2022Convergence of First-Order Methods for Constrained Nonconvex Optimization with Dependent Datacited by 7
  12. 2021Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problemscited by 5
  13. 2022Last-iterate convergence analysis of stochastic momentum methods for neural networkscited by 5
  14. 2025Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillationcited by 5
  15. 2021Fast Convex Quadratic Optimization Solvers with Adaptive Sketching-based Preconditionerscited by 3
  16. 2024Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequalitycited by 3
  17. 2024FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Methodcited by 3
  18. 2025On the Effectiveness of the z-Transform Method in Quadratic Optimizationcited by 3
  19. 2025Learning to optimize with guarantees: a complete characterization of linearly convergent algorithmscited by 3
  20. 2026In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noisecited by 3
  21. 2021Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimizationcited by 2
  22. 2021Gradient Temporal Difference with Momentum: Stability and Convergencecited by 2
  23. 2023Convergence of the Stochastic Heavy Ball Method With Approximate Gradients and/or Block Updatingcited by 2
  24. 2024Parallel Momentum Methods Under Biased Gradient Estimationscited by 2
  25. 2024From Learning to Optimize to Learning Optimization Algorithmscited by 2
  26. 2024Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentumcited by 2
  27. 2025Random feature approximation for general spectral methodscited by 2
  28. 2025Improved Convergence in Parameter-Agnostic Error Feedback through Momentumcited by 2
  29. 2026MGUP: A Momentum-Gradient Alignment Update Policy for Stochastic Optimizationcited by 2
  30. 2021Scaling transition from momentum stochastic gradient descent to plain stochastic gradient descentcited by 1
  31. 2021Improving Adversarial Robustness for Free with Snapshot Ensemblecited by 1
  32. 2022Communication-Efficient {Federated} Learning Using Censored Heavy Ball Descentcited by 1
  33. 2024(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentumcited by 1
  34. 2025Convergence of Momentum-Based Optimization Algorithms with Time-Varying Parameterscited by 1
  35. 2025Analysis of Schedule-Free Nonconvex Optimizationcited by 1
  36. 2026Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalizationcited by 1
  37. 2021On the Generalization of Stochastic Gradient Descent with Momentumcited by 0
  38. 2021Scale Invariant Monte Carlo under Linear Function Approximation with Curvature based step-sizecited by 0

50 most cited of 61 All 61 (JSON)