# Accelerated First-Order Methods: Differential Equations and Lyapunov Functions

arXiv: https://arxiv.org/abs/1903.05671 · HTML: https://mlsanity.com/paper/1903.05671/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Mar 2019.

## Built on by (most cited first, 5 of 5)

- [A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossip](https://mlsanity.com/paper/2106.07644/) (2021)
- [A qualitative difference between gradient flows of convex functions in finite- and infinite-dimensional Hilbert spaces](https://mlsanity.com/paper/2310.17610/) (2023)
- [Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality](https://mlsanity.com/paper/2410.16849/) (2024)
- [A Variational Perspective on High-Resolution ODEs](https://mlsanity.com/paper/2311.02002/) (2023)
- [Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with Momentum](https://mlsanity.com/paper/2203.11992/) (2022)

## Abstract (arXiv)

We develop a theory of accelerated first-order optimization from the viewpoint of differential equations and Lyapunov functions. Building upon the previous work of many researchers, we consider differential equations which model the behavior of accelerated gradient descent. Our main contributions are to provide a general framework for discretizating the differential equations to produce accelerated methods, and to provide physical intuition which helps explain the optimal damping rate. An important novelty is the generality of our approach, which leads to a unified derivation of a wide variety of methods, including versions of Nesterov's accelerated gradient descent, FISTA, and accelerated coordinate descent.