# Fast convex optimization via inertial dynamics with Hessian driven damping

arXiv: https://arxiv.org/abs/1601.07113 · HTML: https://mlsanity.com/paper/1601.07113/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jan 2016.

## Built on by (most cited first, 13 of 13)

- [Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts](https://mlsanity.com/paper/2310.05898/) (2023)
- [Gradient Norm Minimization of Nesterov Acceleration: o(1/k3)](https://mlsanity.com/paper/2209.08862/) (2022)
- [Proximal Subgradient Norm Minimization of ISTA and FISTA](https://mlsanity.com/paper/2211.01610/) (2022)
- [Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality](https://mlsanity.com/paper/2410.16849/) (2024)
- [Inertial Newton Algorithms Avoiding Strict Saddle Points](https://mlsanity.com/paper/2111.04596/) (2021)
- [From Learning to Optimize to Learning Optimization Algorithms](https://mlsanity.com/paper/2405.18222/) (2024)
- [Towards understanding Accelerated Stein Variational Gradient Flow -- Analysis of Generalized Bilinear Kernels for Gaussian target distributions](https://mlsanity.com/paper/2509.04008/) (2025)
- [Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexity](https://mlsanity.com/paper/2306.09694/) (2023)
- [Near-optimal Closed-loop Method via Lyapunov Damping for Convex Optimization](https://mlsanity.com/paper/2311.10053/) (2023)
- [Second Order Drifting Models](https://mlsanity.com/paper/2608.07924/) (2026)
- [Whiplash Gradient Descent Dynamics](https://mlsanity.com/paper/2203.02140/) (2022)
- [Generalized Continuous-Time Models for Nesterov's Accelerated Gradient Methods](https://mlsanity.com/paper/2409.00913/) (2024)
- [A second-order-like optimizer with adaptive gradient scaling for deep learning](https://mlsanity.com/paper/2410.05871/) (2024)

## Abstract (arXiv)

We first study the fast minimization properties of the trajectories of the second-order evolution equation ddotx(t) + fracαt dotx(t) + β nabla2 Phi (x(t))dotx (t) + nabla Phi (x(t)) = 0, where Phi:mathcal H→mathbb R is a smooth convex function acting on a real Hilbert space mathcal H, and α, β are positive parameters. This inertial system combines an isotropic viscous damping which vanishes asymptotically, and a geometrical Hessian driven damping, which makes it naturally related to Newton's and Levenberg-Marquardt methods. For α≥ 3, β >0, along any trajectory, fast convergence of the values Phi(x(t))- minmathcal HPhi =mathcal Oleft(t-2right) is obtained, together with rapid convergence of the gradients nablaPhi(x(t)) to zero. For α>3, just assuming that Phi has minimizers, we show that any trajectory converges weakly to a minimizer of Phi, and  Phi(x(t))-minmathcal HPhi = o(t-2). Strong convergence is established in various practical situations. For the strongly convex case, convergence can be arbitrarily fast depending on the choice of α. More precisely, we have Phi(x(t))- minmathcal HPhi = mathcal O(t^-frac23α). We extend the results to the case of a general proper lower-semicontinuous convex function Phi : mathcal H rightarrow mathbb R cup +∞ . This is based on the fact that the inertial dynamic with Hessian driven damping can be written as a first-order system in time and space. By explicit-implicit time discretization, this opens a gate to new - possibly more rapid - inertial algorithms, expanding the field of FISTA methods for convex structured optimization problems.