# Resource-Aware Discretization of Accelerated Optimization Flows

arXiv: https://arxiv.org/abs/2009.09135 · HTML: https://mlsanity.com/paper/2009.09135/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2020.

## Built on by (most cited first, 2 of 2)

- [A Discrete Variational Derivation of Accelerated Methods in Optimization](https://mlsanity.com/paper/2106.02700/) (2021)
- [A Control Theoretic Framework for Adaptive Gradient Optimizers in Machine Learning](https://mlsanity.com/paper/2206.02034/) (2022)

## Abstract (arXiv)

This paper tackles the problem of discretizing accelerated optimization flows while retaining their convergence properties. Inspired by the success of resource-aware control in developing efficient closed-loop feedback implementations on digital systems, we view the last sampled state of the system as the resource to be aware of. The resulting variable-stepsize discrete-time algorithms retain by design the desired decrease of the Lyapunov certificate of their continuous-time counterparts. Our algorithm design employs various concepts and techniques from resource-aware control that, in the present context, have interesting parallelisms with the discrete-time implementation of optimization algorithms. These include derivative- and performance-based triggers to monitor the evolution of the Lyapunov function as a way of determining the algorithm stepsize, exploiting sampled information to enhance algorithm performance, and employing high-order holds using more accurate integrators of the original dynamics. Throughout the paper, we illustrate our approach on a newly introduced continuous-time dynamics termed heavy-ball dynamics with displaced gradient, but the ideas proposed here have broad applicability to other globally asymptotically stable flows endowed with a Lyapunov certificate.