# On dissipative symplectic integration with applications to gradient-based optimization

arXiv: https://arxiv.org/abs/2004.06840 · HTML: https://mlsanity.com/paper/2004.06840/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2020.

## Built on by (most cited first, 23 of 23)

- [On the Implicit Bias of Adam](https://mlsanity.com/paper/2309.00079/) (2023)
- [On Numerical Integration in Neural Ordinary Differential Equations](https://mlsanity.com/paper/2206.07335/) (2022)
- [Discretization Drift in Two-Player Games](https://mlsanity.com/paper/2105.13922/) (2021)
- [On a continuous time model of gradient descent dynamics and instability in deep learning](https://mlsanity.com/paper/2302.01952/) (2023)
- [Last-Iterate Convergence of Saddle-Point Optimizers via High-Resolution Differential Equations](https://mlsanity.com/paper/2112.13826/) (2021)
- [Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization](https://mlsanity.com/paper/2102.11537/) (2021)
- [Optimization on manifolds: A symplectic approach](https://mlsanity.com/paper/2107.11231/) (2021)
- [Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents](https://mlsanity.com/paper/2203.10592/) (2022)
- [Implicit biases in multitask and continual learning from a backward error analysis perspective](https://mlsanity.com/paper/2311.00235/) (2023)
- [Conformal Symplectic Optimization for Stable Reinforcement Learning](https://mlsanity.com/paper/2412.02291/) (2024)
- [Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning](https://mlsanity.com/paper/2602.16167/) (2026)
- [Revisiting the Effects of Stochasticity for Hamiltonian Samplers](https://mlsanity.com/paper/2106.16200/) (2021)
- [On the connections between optimization algorithms, Lyapunov functions, and differential equations: theory and insights](https://mlsanity.com/paper/2305.08658/) (2023)
- [Fast symplectic integrator for Nesterov-type acceleration method](https://mlsanity.com/paper/2106.07620/) (2021)
- [Continuous-time Analysis for Variational Inequalities: An Overview and Desiderata](https://mlsanity.com/paper/2207.07105/) (2022)
- [SympFormer: Accelerated attention blocks via Inertial Dynamics on Density Manifolds](https://mlsanity.com/paper/2603.16535/) (2026)
- [A More Stable Accelerated Gradient Method Inspired by Continuous-Time Perspective](https://mlsanity.com/paper/2112.04922/) (2021)
- [A Gentle Introduction to Gradient-Based Optimization and Variational Inequalities for Machine Learning](https://mlsanity.com/paper/2309.04877/) (2023)
- [A General Continuous-Time Formulation of Stochastic ADMM and Its Variants](https://mlsanity.com/paper/2404.14358/) (2024)
- [Generalized Continuous-Time Models for Nesterov's Accelerated Gradient Methods](https://mlsanity.com/paper/2409.00913/) (2024)
- [On The Mathematics of the Natural Physics of Optimization](https://mlsanity.com/paper/2604.17645/) (2026)
- [Adaptive Runge-Kutta Step Control Buys Training Loss, Not Generalization: An Honest Compute-Matched Study of RK-Adam Optimizers](https://mlsanity.com/paper/2607.14516/) (2026)
- [When Rates Are Geometric: Rate-Certificate Transfer for Contact Splittings in Optimization](https://mlsanity.com/paper/2607.23642/) (2026)

## Abstract (arXiv)

Recently, continuous-time dynamical systems have proved useful in providing conceptual and quantitative insights into gradient-based optimization, widely used in modern machine learning and statistics. An important question that arises in this line of work is how to discretize the system in such a way that its stability and rates of convergence are preserved. In this paper we propose a geometric framework in which such discretizations can be realized systematically, enabling the derivation of "rate-matching" algorithms without the need for a discrete convergence analysis. More specifically, we show that a generalization of symplectic integrators to nonconservative and in particular dissipative Hamiltonian systems is able to preserve rates of convergence up to a controlled error. Moreover, such methods preserve a shadow Hamiltonian despite the absence of a conservation law, extending key results of symplectic integrators to nonconservative cases. Our arguments rely on a combination of backward error analysis with fundamental results from symplectic geometry. We stress that although the original motivation for this work was the application to optimization, where dissipative systems play a natural role, they are fully general and not only provide a differential geometric framework for dissipative Hamiltonian systems but also substantially extend the theory of structure-preserving integration.