# A Control-Theoretic Perspective on Optimal High-Order Optimization

arXiv: https://arxiv.org/abs/1912.07168 · HTML: https://mlsanity.com/paper/1912.07168/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2019.

## Built on by (most cited first, 5 of 5)

- [Perseus: A Simple and Optimal High-Order Method for Variational Inequalities](https://mlsanity.com/paper/2205.03202/) (2022)
- [Second-Order Min-Max Optimization with Lazy Hessians](https://mlsanity.com/paper/2410.09568/) (2024)
- [Neural Observer with Lyapunov Stability Guarantee for Uncertain Nonlinear Systems](https://mlsanity.com/paper/2208.13006/) (2022)
- [COMPASS: Context-Modulated PID Attention Steering System for Hallucination Mitigation](https://mlsanity.com/paper/2511.14776/) (2025)
- [Near-optimal Closed-loop Method via Lyapunov Damping for Convex Optimization](https://mlsanity.com/paper/2311.10053/) (2023)

## Abstract (arXiv)

We provide a control-theoretic perspective on optimal tensor algorithms for minimizing a convex function in a finite-dimensional Euclidean space. Given a function Phi: ℝd rightarrow ℝ that is convex and twice continuously differentiable, we study a closed-loop control system that is governed by the operators nabla Phi and nabla2 Phi together with a feedback control law λ(·) satisfying the algebraic equation (λ(t))p|nablaPhi(x(t))|p-1 = θ for some θ ∈ (0, 1). Our first contribution is to prove the existence and uniqueness of a local solution to this system via the Banach fixed-point theorem. We present a simple yet nontrivial Lyapunov function that allows us to establish the existence and uniqueness of a global solution under certain regularity conditions and analyze the convergence properties of trajectories. The rate of convergence is O(1/t(3p+1)/2) in terms of objective function gap and O(1/t3p) in terms of squared gradient norm. Our second contribution is to provide two algorithmic frameworks obtained from discretization of our continuous-time system, one of which generalizes the large-step A-HPE framework and the other of which leads to a new optimal p-th order tensor algorithm. While our discrete-time analysis can be seen as a simplification and generalization of~citetMonteiro-2013-Accelerated, it is largely motivated by the aforementioned continuous-time analysis, demonstrating the fundamental role that the feedback control plays in optimal acceleration and the clear advantage that the continuous-time perspective brings to algorithmic design. A highlight of our analysis is that we show that all of the p-th order optimal tensor algorithms that we discuss minimize the squared gradient norm at a rate of O(k-3p), which complements the recent analysis.