# A consistently adaptive trust-region method

arXiv: https://arxiv.org/abs/2408.01874 · HTML: https://mlsanity.com/paper/2408.01874/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Aug 2024.

## Built on by (most cited first, 1 of 1)

- [Efficient Second-Order Neural Network Optimization via Adaptive Trust Region Methods](https://mlsanity.com/paper/2410.02293/) (2024)

## Abstract (arXiv)

Adaptive trust-region methods attempt to maintain strong convergence guarantees without depending on conservative estimates of problem properties such as Lipschitz constants. However, on close inspection, one can show existing adaptive trust-region methods have theoretical guarantees with severely suboptimal dependence on problem properties such as the Lipschitz constant of the Hessian. For example, TRACE developed by Curtis et al. obtains a O(Deltaf L3/2 ε-3/2) + tildeO(1) iteration bound where L is the Lipschitz constant of the Hessian. Compared with the optimal O(Deltaf L1/2 ε-3/2) bound this is suboptimal with respect to L. We present the first adaptive trust-region method which circumvents this issue and requires at most O( Deltaf L1/2 ε-3/2) + tildeO(1) iterations to find an ε-approximate stationary point, matching the optimal iteration bound up to an additive logarithmic term. Our method is a simple variant of a classic trust-region method and in our experiments performs competitively with both ARC and a classical trust-region method.