mlsanityData to 7 Oct 2026

Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization

arXiv 2102.11537 ↗First posted 23 Feb 2021math.OC cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Viewing optimization methods as numerical integrators for ordinary differential equations (ODEs) provides a thought-provoking modern framework for studying accelerated first-order optimizers. In this literature, acceleration is often supposed to be linked to the quality of the integrator (accuracy, energy preservation, symplecticity). In this work, we propose a novel ordinary differential equation that questions this connection: both the explicit and the semi-implicit (a.k.a symplectic) Euler discretizations on this ODE lead to an accelerated algorithm for convex programming. Although semi-implicit methods are well-known in numerical analysis to enjoy many desirable features for the integration of physical systems, our findings show that these properties do not necessarily relate to acceleration.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 29 of its 54 references that are in the map. First posted on arXiv 23 Feb 2021.

Builds on

29 of 54 references.

References · 29

  1. 2015A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insightscited by 203
  2. 2016A Variational Perspective on Accelerated Methods in Optimizationcited by 108
  3. 2014Analysis and Design of Optimization Algorithms via Integral Quadratic Constraintscited by 105
  4. 2019On Empirical Comparisons of Optimizers for Deep Learningcited by 99
  5. 2018Understanding the Acceleration Phenomenon via High-Resolution Differential Equationscited by 68
  6. 2018Quasi-hyperbolic momentum and Adam for deep learningcited by 47
  7. 2019Understanding the Role of Momentum in Stochastic Gradient Methodscited by 42
  8. 2019A Dynamical Systems Perspective on Nesterov Accelerationcited by 33
  9. 2016A Lyapunov Analysis of Momentum Methods in Optimizationcited by 28
  10. 2019Conformal Symplectic and Relativistic Optimizationcited by 28
  11. 2019Acceleration via Symplectic Discretization of High-Resolution Differential Equationscited by 27
  12. 2020On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settingscited by 27
17 more
  1. 2020Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectivescited by 27
  2. 2019A Continuous-time Perspective for Modeling Acceleration in Riemannian Optimizationcited by 25
  3. 2015From Averaging to Acceleration, There is Only a Step-sizecited by 24
  4. 2020On dissipative symplectic integration with applications to gradient-based optimizationcited by 23
  5. 2017Dissipativity Theory for Nesterov's Accelerated Methodcited by 22
  6. 2018Direct Runge-Kutta Discretization Achieves Accelerationcited by 21
  7. 2017Analysis of Optimization Algorithms via Integral Quadratic Constraints: Nonstrongly Convex Problemscited by 19
  8. 2019Generalized Momentum-Based Methods: A Hamiltonian Perspectivecited by 12
  9. 2020The connections between Lyapunov functions for some optimization algorithms and differential equationscited by 11
  10. 2019The Role of Memory in Stochastic Optimizationcited by 10
  11. 2017Accelerated Extra-Gradient Descent: A Novel Accelerated First-Order Methodcited by 8
  12. 2018On the Curved Geometry of Accelerated Optimizationcited by 6
  13. 2019Shadowing Properties of Optimization Algorithmscited by 6
  14. 2019An extension of the second order dynamical system that models Nesterov's convex gradient methodcited by 5
  15. 2019Contractivity of Runge-Kutta methods for convex gradient systemscited by 4
  16. 2019Acceleration in First Order Quasi-strongly Convex Optimization by ODE Discretizationcited by 1
  17. 2020The long time behavior and the rate of convergence of symplectic convex algorithms obtained via splitting discretizations of inertial damping systemscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1; 2024: 1; 2025: 1.

6 papers.

  1. 2022From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descentcited by 5
  2. 2022Revisiting the acceleration phenomenon via high-resolution differential equationscited by 5
  3. 2024ODE-based Learning to Optimizecited by 4
  4. 2023Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexitycited by 1
  5. 2025Data Generation without Function Estimationcited by 1
  6. 2022Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networkscited by 0