# Fast symplectic integrator for Nesterov-type acceleration method

arXiv: https://arxiv.org/abs/2106.07620 · HTML: https://mlsanity.com/paper/2106.07620/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 49 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights](https://mlsanity.com/paper/1503.01243/) (2015)
- [A Variational Perspective on Accelerated Methods in Optimization](https://mlsanity.com/paper/1603.04245/) (2016)
- [Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints](https://mlsanity.com/paper/1408.3595/) (2014)
- [A Survey of Optimization Methods from a Machine Learning Perspective](https://mlsanity.com/paper/1906.06821/) (2019)
- [Acceleration via Symplectic Discretization of High-Resolution Differential Equations](https://mlsanity.com/paper/1902.03694/) (2019)
- [On dissipative symplectic integration with applications to gradient-based optimization](https://mlsanity.com/paper/2004.06840/) (2020)
- [Dissipativity Theory for Nesterov's Accelerated Method](https://mlsanity.com/paper/1706.04381/) (2017)
- [Direct Runge-Kutta Discretization Achieves Acceleration](https://mlsanity.com/paper/1805.00521/) (2018)
- [Analysis of Optimization Algorithms via Integral Quadratic Constraints: Nonstrongly Convex Problems](https://mlsanity.com/paper/1705.03615/) (2017)
- [Regularized Nonlinear Acceleration](https://mlsanity.com/paper/1606.04133/) (2016)
- [Lyapunov Theory for Discrete Time Systems](https://mlsanity.com/paper/1809.05289/) (2018)
- [Generalized Momentum-Based Methods: A Hamiltonian Perspective](https://mlsanity.com/paper/1906.00436/) (2019)
- [Variational Optimization on Lie Groups, with Examples of Leading (Generalized) Eigenvalue Problems](https://mlsanity.com/paper/2001.10006/) (2020)
- [Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3](https://mlsanity.com/paper/1706.05671/) (2017)
- [Accelerating Rescaled Gradient Descent: Fast Optimization of Smooth Functions](https://mlsanity.com/paper/1902.08825/) (2019)
- [On the Curved Geometry of Accelerated Optimization](https://mlsanity.com/paper/1812.04634/) (2018)
- [Bregman dynamics, contact transformations and convex optimization](https://mlsanity.com/paper/1912.02928/) (2019)

## Built on by (most cited first, 1 of 1)

- [SympFormer: Accelerated attention blocks via Inertial Dynamics on Density Manifolds](https://mlsanity.com/paper/2603.16535/) (2026)

## Abstract (arXiv)

In this paper, explicit stable integrators based on symplectic and contact geometries are proposed for a non-autonomous ordinarily differential equation (ODE) found in improving convergence rate of Nesterov's accelerated gradient method. Symplectic geometry is known to be suitable for describing Hamiltonian mechanics, and contact geometry is known as an odd-dimensional counterpart of symplectic geometry. Moreover, a procedure, called symplectization, is a known way to construct a symplectic manifold from a contact manifold, yielding Hamiltonian systems from contact ones. It is found in this paper that a previously investigated non-autonomous ODE can be written as a contact Hamiltonian system. Then, by symplectization of a non-autonomous contact Hamiltonian vector field expressing the non-autonomous ODE, novel symplectic integrators are derived. Because the proposed symplectic integrators preserve hidden symplectic and contact structures in the ODE, they should be more stable than the Runge-Kutta method. Numerical experiments demonstrate that, as expected, the second-order symplectic integrator is stable and high convergence rates are achieved.