# Whiplash Gradient Descent Dynamics

arXiv: https://arxiv.org/abs/2203.02140 · HTML: https://mlsanity.com/paper/2203.02140/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 43 references that are in the map.
- First posted on arXiv 4 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition](https://mlsanity.com/paper/1608.04636/) (2016)
- [How to Escape Saddle Points Efficiently](https://mlsanity.com/paper/1703.00887/) (2017)
- [A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights](https://mlsanity.com/paper/1503.01243/) (2015)
- [A Variational Perspective on Accelerated Methods in Optimization](https://mlsanity.com/paper/1603.04245/) (2016)
- [Acceleration via Symplectic Discretization of High-Resolution Differential Equations](https://mlsanity.com/paper/1902.03694/) (2019)
- [Exponentially vanishing sub-optimal local minima in multilayer neural networks](https://mlsanity.com/paper/1702.05777/) (2017)
- [Fast convex optimization via inertial dynamics with Hessian driven damping](https://mlsanity.com/paper/1601.07113/) (2016)
- [Optimal convergence rates for Nesterov acceleration](https://mlsanity.com/paper/1805.05719/) (2018)
- [Fast optimization via inertial dynamics with closed-loop damping](https://mlsanity.com/paper/2008.02261/) (2020)
- [Drill the Cork of Information Bottleneck by Inputting the Most Important Data](https://mlsanity.com/paper/2105.07181/) (2021)
- [A closed loop gradient descent algorithm applied to Rosenbrock's function](https://mlsanity.com/paper/2108.12883/) (2021)

## Abstract (arXiv)

In this paper, we propose the Whiplash Inertial Gradient dynamics, a closed-loop optimization method that utilises gradient information, to find the minima of a cost function in finite-dimensional settings. We introduce the symplectic asymptotic convergence analysis for the Whiplash system for convex functions. We also introduce relaxation sequences to explain the non-classical nature of the algorithm and an exploring heuristic variant of the Whiplash algorithm to escape saddle points, deterministically. We study the algorithm's performance for various costs and provide a practical methodology for analyzing convergence rates using integral constraint bounds and a novel Lyapunov rate method. Our results demonstrate polynomial and exponential rates of convergence for quadratic cost functions.