mlsanityData to 7 Oct 2026

A Dynamical Systems Perspective on Nesterov Acceleration

arXiv 1905.07436 ↗First posted 17 May 2019math.OC cs.LG cs.SYcited by 33

Not on the map. Why?

Abstract (arXiv)

We present a dynamical system framework for understanding Nesterov's accelerated gradient method. In contrast to earlier work, our derivation does not rely on a vanishing step size argument. We show that Nesterov acceleration arises from discretizing an ordinary differential equation with a semi-implicit Euler integration scheme. We analyze both the underlying differential equation as well as the discretization to obtain insights into the phenomenon of acceleration. The analysis suggests that a curvature-dependent damping term lies at the heart of the phenomenon. We further establish connections between the discretized and the continuous-time dynamics.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 8; 2023: 10; 2024: 6; 2025: 3; 2026: 1.

33 papers.

Sort:

  1. 2021A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossipcited by 15
  2. 2021On Constraints in First-Order Optimization: A View from Non-Smooth Dynamical Systemscited by 13
  3. 2024DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reductioncited by 10
  4. 2023Accelerating optimization over the space of probability measurescited by 9
  5. 2022Gradient Norm Minimization of Nesterov Acceleration: o(1/k3)cited by 8
  6. 2021Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimizationcited by 6
  7. 2022Revisiting the acceleration phenomenon via high-resolution differential equationscited by 5
  8. 2023Accelerated First-Order Optimization under Nonlinear Constraintscited by 5
  9. 2022Neuroevolution of Physics-Informed Neural Nets: Benchmark Problems and Comparative Resultscited by 4
  10. 2023A Nonstochastic Control Approach to Optimizationcited by 4
  11. 2021A Contraction Theory Approach to Optimization Algorithms from Acceleration Flowscited by 3
  12. 2024Distributed Event-Based Learning via ADMMcited by 3
21 more
  1. 2025Continuous-Time Analysis of Heavy Ball Momentum in Min-Max Gamescited by 3
  2. 2022Understanding A Class of Decentralized and Federated Optimization Algorithms: A Multi-Rate Feedback Control Perspectivecited by 2
  3. 2022NAG-GS: Semi-Implicit, Accelerated and Robust Stochastic Optimizercited by 2
  4. 2023On the connections between optimization algorithms, Lyapunov functions, and differential equations: theory and insightscited by 2
  5. 2023Accelerated Policy Gradient: On the Convergence Rates of the Nesterov Momentum for Reinforcement Learningcited by 2
  6. 2023A Variational Perspective on High-Resolution ODEscited by 2
  7. 2024Accelerated forward-backward and Douglas-Rachford splitting dynamicscited by 2
  8. 2024Is All Learning (Natural) Gradient Descent?cited by 2
  9. 2021A closed loop gradient descent algorithm applied to Rosenbrock's functioncited by 1
  10. 2022Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with Momentumcited by 1
  11. 2022Continuous-time Analysis for Variational Inequalities: An Overview and Desideratacited by 1
  12. 2022Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithmscited by 1
  13. 2023On Underdamped Nesterov's Accelerationcited by 1
  14. 2023Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexitycited by 1
  15. 2023Towards Hyperparameter-Agnostic DNN Training via Dynamical System Insightscited by 1
  16. 2024Error estimates between SGD with momentum and underdamped Langevin diffusioncited by 1
  17. 2025Learning by solving differential equationscited by 1
  18. 2023Accelerated Optimization Landscape of Linear-Quadratic Regulatorcited by 0
  19. 2024Generalized Continuous-Time Models for Nesterov's Accelerated Gradient Methodscited by 0
  20. 2025A Systems-Theoretic View on the Convergence of Algorithms under Disturbancescited by 0
  21. 2026When Rates Are Geometric: Rate-Certificate Transfer for Contact Splittings in Optimizationcited by 0