# Warped geometric information on the optimisation of Euclidean functions

arXiv: https://arxiv.org/abs/2308.08305 · HTML: https://mlsanity.com/paper/2308.08305/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 54 references that are in the map.
- First posted on arXiv 16 Aug 2023.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Riemannian Adaptive Optimization Methods](https://mlsanity.com/paper/1810.00760/) (2018)
- [Langevin diffusions and the Metropolis-adjusted Langevin algorithm](https://mlsanity.com/paper/1309.2983/) (2013)
- [Conformal Symplectic and Relativistic Optimization](https://mlsanity.com/paper/1903.04100/) (2019)
- [Metrics for Probabilistic Geometries](https://mlsanity.com/paper/1411.7432/) (2014)
- [Accelerating Natural Gradient with Higher-Order Invariance](https://mlsanity.com/paper/1803.01273/) (2018)
- [Optimal Preconditioning and Fisher Adaptive Langevin Sampling](https://mlsanity.com/paper/2305.14442/) (2023)
- [Riemannian conjugate gradient methods: General framework and specific algorithms with convergence analyses](https://mlsanity.com/paper/2112.02572/) (2021)
- [Lagrangian Manifold Monte Carlo on Monge Patches](https://mlsanity.com/paper/2202.00755/) (2022)
- [Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metrics](https://mlsanity.com/paper/2303.05101/) (2023)
- [Laplace approximation and the natural gradient for Gaussian process regression with the heteroscedastic Student-t model](https://mlsanity.com/paper/1712.07437/) (2017)
- [A Variational Formulation of Accelerated Optimization on Riemannian Manifolds](https://mlsanity.com/paper/2101.06552/) (2021)
- [Sufficient Descent Riemannian Conjugate Gradient Method](https://mlsanity.com/paper/2009.01451/) (2020)
- [Accelerated Optimization on Riemannian Manifolds via Projected Variational Integrators](https://mlsanity.com/paper/2201.02904/) (2022)

## Built on by (most cited first, 3 of 3)

- [Riemannian Laplace Approximation with the Fisher Metric](https://mlsanity.com/paper/2311.02766/) (2023)
- [What's Inside Your Diffusion Model? A Score-Based Riemannian Metric to Explore the Data Manifold](https://mlsanity.com/paper/2505.11128/) (2025)
- [Geodesic Slice Sampler for Multimodal Distributions with Strong Curvature](https://mlsanity.com/paper/2502.21190/) (2025)

## Abstract (arXiv)

We consider the fundamental task of optimising a real-valued function defined in a potentially high-dimensional Euclidean space, such as the loss function in many machine-learning tasks or the logarithm of the probability distribution in statistical inference. We use Riemannian geometry notions to redefine the optimisation problem of a function on the Euclidean space to a Riemannian manifold with a warped metric, and then find the function's optimum along this manifold. The warped metric chosen for the search domain induces a computational friendly metric-tensor for which optimal search directions associated with geodesic curves on the manifold becomes easier to compute. Performing optimization along geodesics is known to be generally infeasible, yet we show that in this specific manifold we can analytically derive Taylor approximations up to third-order. In general these approximations to the geodesic curve will not lie on the manifold, however we construct suitable retraction maps to pull them back onto the manifold. Therefore, we can efficiently optimize along the approximate geodesic curves. We cover the related theory, describe a practical optimization algorithm and empirically evaluate it on a collection of challenging optimisation benchmarks. Our proposed algorithm, using 3rd-order approximation of geodesics, tends to outperform standard Euclidean gradient-based counterparts in term of number of iterations until convergence.