# Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization

arXiv: https://arxiv.org/abs/2102.11537 · HTML: https://mlsanity.com/paper/2102.11537/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 29 of 29)

- [A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights](https://mlsanity.com/paper/1503.01243/) (2015)
- [A Variational Perspective on Accelerated Methods in Optimization](https://mlsanity.com/paper/1603.04245/) (2016)
- [Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints](https://mlsanity.com/paper/1408.3595/) (2014)
- [On Empirical Comparisons of Optimizers for Deep Learning](https://mlsanity.com/paper/1910.05446/) (2019)
- [Understanding the Acceleration Phenomenon via High-Resolution Differential Equations](https://mlsanity.com/paper/1810.08907/) (2018)
- [Quasi-hyperbolic momentum and Adam for deep learning](https://mlsanity.com/paper/1810.06801/) (2018)
- [Understanding the Role of Momentum in Stochastic Gradient Methods](https://mlsanity.com/paper/1910.13962/) (2019)
- [A Dynamical Systems Perspective on Nesterov Acceleration](https://mlsanity.com/paper/1905.07436/) (2019)
- [Conformal Symplectic and Relativistic Optimization](https://mlsanity.com/paper/1903.04100/) (2019)
- [A Lyapunov Analysis of Momentum Methods in Optimization](https://mlsanity.com/paper/1611.02635/) (2016)
- [Acceleration via Symplectic Discretization of High-Resolution Differential Equations](https://mlsanity.com/paper/1902.03694/) (2019)
- [On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings](https://mlsanity.com/paper/2002.12414/) (2020)
- [Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives](https://mlsanity.com/paper/2002.12493/) (2020)
- [A Continuous-time Perspective for Modeling Acceleration in Riemannian Optimization](https://mlsanity.com/paper/1910.10782/) (2019)
- [From Averaging to Acceleration, There is Only a Step-size](https://mlsanity.com/paper/1504.01577/) (2015)
- [On dissipative symplectic integration with applications to gradient-based optimization](https://mlsanity.com/paper/2004.06840/) (2020)
- [Dissipativity Theory for Nesterov's Accelerated Method](https://mlsanity.com/paper/1706.04381/) (2017)
- [Direct Runge-Kutta Discretization Achieves Acceleration](https://mlsanity.com/paper/1805.00521/) (2018)
- [Analysis of Optimization Algorithms via Integral Quadratic Constraints: Nonstrongly Convex Problems](https://mlsanity.com/paper/1705.03615/) (2017)
- [Generalized Momentum-Based Methods: A Hamiltonian Perspective](https://mlsanity.com/paper/1906.00436/) (2019)
- [The connections between Lyapunov functions for some optimization algorithms and differential equations](https://mlsanity.com/paper/2009.00673/) (2020)
- [The Role of Memory in Stochastic Optimization](https://mlsanity.com/paper/1907.01678/) (2019)
- [Accelerated Extra-Gradient Descent: A Novel Accelerated First-Order Method](https://mlsanity.com/paper/1706.04680/) (2017)
- [On the Curved Geometry of Accelerated Optimization](https://mlsanity.com/paper/1812.04634/) (2018)
- [Shadowing Properties of Optimization Algorithms](https://mlsanity.com/paper/1911.05206/) (2019)
- [An extension of the second order dynamical system that models Nesterov's convex gradient method](https://mlsanity.com/paper/1908.02574/) (2019)
- [Contractivity of Runge-Kutta methods for convex gradient systems](https://mlsanity.com/paper/1909.09971/) (2019)
- [Acceleration in First Order Quasi-strongly Convex Optimization by ODE Discretization](https://mlsanity.com/paper/1905.12436/) (2019)
- [The long time behavior and the rate of convergence of symplectic convex algorithms obtained via splitting discretizations of inertial damping systems](https://mlsanity.com/paper/2001.10831/) (2020)

## Built on by (most cited first, 6 of 6)

- [From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descent](https://mlsanity.com/paper/2210.06705/) (2022)
- [Revisiting the acceleration phenomenon via high-resolution differential equations](https://mlsanity.com/paper/2212.05700/) (2022)
- [ODE-based Learning to Optimize](https://mlsanity.com/paper/2406.02006/) (2024)
- [Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexity](https://mlsanity.com/paper/2306.09694/) (2023)
- [Data Generation without Function Estimation](https://mlsanity.com/paper/2507.08239/) (2025)
- [Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks](https://mlsanity.com/paper/2208.03941/) (2022)

## Abstract (arXiv)

Viewing optimization methods as numerical integrators for ordinary differential equations (ODEs) provides a thought-provoking modern framework for studying accelerated first-order optimizers. In this literature, acceleration is often supposed to be linked to the quality of the integrator (accuracy, energy preservation, symplecticity). In this work, we propose a novel ordinary differential equation that questions this connection: both the explicit and the semi-implicit (a.k.a symplectic) Euler discretizations on this ODE lead to an accelerated algorithm for convex programming. Although semi-implicit methods are well-known in numerical analysis to enjoy many desirable features for the integration of physical systems, our findings show that these properties do not necessarily relate to acceleration.