Whiplash Gradient Descent Dynamics
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose the Whiplash Inertial Gradient dynamics, a closed-loop optimization method that utilises gradient information, to find the minima of a cost function in finite-dimensional settings. We introduce the symplectic asymptotic convergence analysis for the Whiplash system for convex functions. We also introduce relaxation sequences to explain the non-classical nature of the algorithm and an exploring heuristic variant of the Whiplash algorithm to escape saddle points, deterministically. We study the algorithm's performance for various costs and provide a practical methodology for analyzing convergence rates using integral constraint bounds and a novel Lyapunov rate method. Our results demonstrate polynomial and exponential rates of convergence for quadratic cost functions.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 43 references that are in the map. First posted on arXiv 4 Mar 2022.
Builds on
12 of 43 references.
References · 12
- 2021Convex optimizationcited by 2,479
- 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
- 2017How to Escape Saddle Points Efficientlycited by 284
- 2015A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insightscited by 203
- 2016A Variational Perspective on Accelerated Methods in Optimizationcited by 108
- 2019Acceleration via Symplectic Discretization of High-Resolution Differential Equationscited by 27
- 2017Exponentially vanishing sub-optimal local minima in multilayer neural networkscited by 19
- 2016Fast convex optimization via inertial dynamics with Hessian driven dampingcited by 13
- 2018Optimal convergence rates for Nesterov accelerationcited by 8
- 2020Fast optimization via inertial dynamics with closed-loop dampingcited by 4
- 2021Drill the Cork of Information Bottleneck by Inputting the Most Important Datacited by 1
- 2021A closed loop gradient descent algorithm applied to Rosenbrock's functioncited by 1
Built on by
Not cited yet.