# Conformal Symplectic and Relativistic Optimization

arXiv: https://arxiv.org/abs/1903.04100 · HTML: https://mlsanity.com/paper/1903.04100/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Mar 2019.

## Built on by (most cited first, 29 of 29)

- [Discretization Drift in Two-Player Games](https://mlsanity.com/paper/2105.13922/) (2021)
- [On Constraints in First-Order Optimization: A View from Non-Smooth Dynamical Systems](https://mlsanity.com/paper/2107.08225/) (2021)
- [On a continuous time model of gradient descent dynamics and instability in deep learning](https://mlsanity.com/paper/2302.01952/) (2023)
- [Born-Infeld (BI) for AI: Energy-Conserving Descent (ECD) for Optimization](https://mlsanity.com/paper/2201.11137/) (2022)
- [Accelerating optimization over the space of probability measures](https://mlsanity.com/paper/2310.04006/) (2023)
- [Generalisation under gradient descent via deterministic PAC-Bayes](https://mlsanity.com/paper/2209.02525/) (2022)
- [Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization](https://mlsanity.com/paper/2102.11537/) (2021)
- [Optimization on manifolds: A symplectic approach](https://mlsanity.com/paper/2107.11231/) (2021)
- [Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents](https://mlsanity.com/paper/2203.10592/) (2022)
- [Accelerated First-Order Optimization under Nonlinear Constraints](https://mlsanity.com/paper/2302.00316/) (2023)
- [Conformal Symplectic Optimization for Stable Reinforcement Learning](https://mlsanity.com/paper/2412.02291/) (2024)
- [A Contraction Theory Approach to Optimization Algorithms from Acceleration Flows](https://mlsanity.com/paper/2105.08832/) (2021)
- [Revisiting the Effects of Stochasticity for Hamiltonian Samplers](https://mlsanity.com/paper/2106.16200/) (2021)
- [Stochastic Subgradient Methods with Guaranteed Global Stability in Nonsmooth Nonconvex Optimization](https://mlsanity.com/paper/2307.10053/) (2023)
- [Warped geometric information on the optimisation of Euclidean functions](https://mlsanity.com/paper/2308.08305/) (2023)
- [NEO: Non Equilibrium Sampling on the Orbit of a Deterministic Transform](https://mlsanity.com/paper/2103.10943/) (2021)
- [A Physics-Inspired Optimizer: Velocity Regularized Adam](https://mlsanity.com/paper/2505.13196/) (2025)
- [Learning by solving differential equations](https://mlsanity.com/paper/2505.13397/) (2025)
- [SympFormer: Accelerated attention blocks via Inertial Dynamics on Density Manifolds](https://mlsanity.com/paper/2603.16535/) (2026)
- [A More Stable Accelerated Gradient Method Inspired by Continuous-Time Perspective](https://mlsanity.com/paper/2112.04922/) (2021)
- [Understanding Accelerated Gradient Methods: Lyapunov Analyses and Hamiltonian Assisted Interpretations](https://mlsanity.com/paper/2304.10063/) (2023)
- [Accelerated Optimization Landscape of Linear-Quadratic Regulator](https://mlsanity.com/paper/2307.03590/) (2023)
- [Repelling-Attracting Hamiltonian Monte Carlo](https://mlsanity.com/paper/2403.04607/) (2024)
- [When Descent Is Too Stable: Event-Triggered Hamiltonian Learning to Optimize](https://mlsanity.com/paper/2605.06868/) (2026)
- [Causal Atlases from Entropic Inference: Bayesian Networks beyond Optimal DAGs](https://mlsanity.com/paper/2606.06440/) (2026)
- [Adaptive Runge-Kutta Step Control Buys Training Loss, Not Generalization: An Honest Compute-Matched Study of RK-Adam Optimizers](https://mlsanity.com/paper/2607.14516/) (2026)
- [When Rates Are Geometric: Rate-Certificate Transfer for Contact Splittings in Optimization](https://mlsanity.com/paper/2607.23642/) (2026)
- [Variational Structure at the Edge of Stability](https://mlsanity.com/paper/2608.21660/) (2026)
- [Hamiltonian Metric Learning and Energy-Based Training: A Dissipative Geometric Framework for Optimization](https://mlsanity.com/paper/2610.04969/) (2026)

## Abstract (arXiv)

Arguably, the two most popular accelerated or momentum-based optimization methods in machine learning are Nesterov's accelerated gradient and Polyaks's heavy ball, both corresponding to different discretizations of a particular second order differential equation with friction. Such connections with continuous-time dynamical systems have been instrumental in demystifying acceleration phenomena in optimization. Here we study structure-preserving discretizations for a certain class of dissipative (conformal) Hamiltonian systems, allowing us to analyze the symplectic structure of both Nesterov and heavy ball, besides providing several new insights into these methods. Moreover, we propose a new algorithm based on a dissipative relativistic system that normalizes the momentum and may result in more stable/faster optimization. Importantly, such a method generalizes both Nesterov and heavy ball, each being recovered as distinct limiting cases, and has potential advantages at no additional cost.