# A Dynamical Systems Perspective on Nesterov Acceleration

arXiv: https://arxiv.org/abs/1905.07436 · HTML: https://mlsanity.com/paper/1905.07436/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2019.

## Built on by (most cited first, 30 of 33)

- [A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossip](https://mlsanity.com/paper/2106.07644/) (2021)
- [On Constraints in First-Order Optimization: A View from Non-Smooth Dynamical Systems](https://mlsanity.com/paper/2107.08225/) (2021)
- [DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction](https://mlsanity.com/paper/2410.03883/) (2024)
- [Accelerating optimization over the space of probability measures](https://mlsanity.com/paper/2310.04006/) (2023)
- [Gradient Norm Minimization of Nesterov Acceleration: o(1/k3)](https://mlsanity.com/paper/2209.08862/) (2022)
- [Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization](https://mlsanity.com/paper/2102.11537/) (2021)
- [Revisiting the acceleration phenomenon via high-resolution differential equations](https://mlsanity.com/paper/2212.05700/) (2022)
- [Accelerated First-Order Optimization under Nonlinear Constraints](https://mlsanity.com/paper/2302.00316/) (2023)
- [Neuroevolution of Physics-Informed Neural Nets: Benchmark Problems and Comparative Results](https://mlsanity.com/paper/2212.07624/) (2022)
- [A Nonstochastic Control Approach to Optimization](https://mlsanity.com/paper/2301.07902/) (2023)
- [Accelerated Policy Gradient: On the Convergence Rates of the Nesterov Momentum for Reinforcement Learning](https://mlsanity.com/paper/2310.11897/) (2023)
- [A Contraction Theory Approach to Optimization Algorithms from Acceleration Flows](https://mlsanity.com/paper/2105.08832/) (2021)
- [Distributed Event-Based Learning via ADMM](https://mlsanity.com/paper/2405.10618/) (2024)
- [Continuous-Time Analysis of Heavy Ball Momentum in Min-Max Games](https://mlsanity.com/paper/2505.19537/) (2025)
- [Understanding A Class of Decentralized and Federated Optimization Algorithms: A Multi-Rate Feedback Control Perspective](https://mlsanity.com/paper/2204.12663/) (2022)
- [NAG-GS: Semi-Implicit, Accelerated and Robust Stochastic Optimizer](https://mlsanity.com/paper/2209.14937/) (2022)
- [On the connections between optimization algorithms, Lyapunov functions, and differential equations: theory and insights](https://mlsanity.com/paper/2305.08658/) (2023)
- [A Variational Perspective on High-Resolution ODEs](https://mlsanity.com/paper/2311.02002/) (2023)
- [Accelerated forward-backward and Douglas-Rachford splitting dynamics](https://mlsanity.com/paper/2407.20620/) (2024)
- [Is All Learning (Natural) Gradient Descent?](https://mlsanity.com/paper/2409.16422/) (2024)
- [A closed loop gradient descent algorithm applied to Rosenbrock's function](https://mlsanity.com/paper/2108.12883/) (2021)
- [Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with Momentum](https://mlsanity.com/paper/2203.11992/) (2022)
- [Continuous-time Analysis for Variational Inequalities: An Overview and Desiderata](https://mlsanity.com/paper/2207.07105/) (2022)
- [Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithms](https://mlsanity.com/paper/2209.11920/) (2022)
- [On Underdamped Nesterov's Acceleration](https://mlsanity.com/paper/2304.14642/) (2023)
- [Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexity](https://mlsanity.com/paper/2306.09694/) (2023)
- [Towards Hyperparameter-Agnostic DNN Training via Dynamical System Insights](https://mlsanity.com/paper/2310.13901/) (2023)
- [Error estimates between SGD with momentum and underdamped Langevin diffusion](https://mlsanity.com/paper/2410.17297/) (2024)
- [Learning by solving differential equations](https://mlsanity.com/paper/2505.13397/) (2025)
- [Accelerated Optimization Landscape of Linear-Quadratic Regulator](https://mlsanity.com/paper/2307.03590/) (2023)

## Abstract (arXiv)

We present a dynamical system framework for understanding Nesterov's accelerated gradient method. In contrast to earlier work, our derivation does not rely on a vanishing step size argument. We show that Nesterov acceleration arises from discretizing an ordinary differential equation with a semi-implicit Euler integration scheme. We analyze both the underlying differential equation as well as the discretization to obtain insights into the phenomenon of acceleration. The analysis suggests that a curvature-dependent damping term lies at the heart of the phenomenon. We further establish connections between the discretized and the continuous-time dynamics.