# Geometric Optimisation on Manifolds with Applications to Deep Learning

arXiv: https://arxiv.org/abs/2203.04794 · HTML: https://mlsanity.com/paper/2203.04794/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2022.

## Built on by (most cited first, 1 of 1)

- [Lie Group Decompositions for Equivariant Neural Networks](https://mlsanity.com/paper/2310.11366/) (2023)

## Abstract (arXiv)

We design and implement a Python library to help the non-expert using all these powerful tools in a way that is efficient, extensible, and simple to incorporate into the workflow of the data scientist, practitioner, and applied researcher. The algorithms implemented in this library have been designed with usability and GPU efficiency in mind, and they can be added to any PyTorch model with just one extra line of code. We showcase the effectiveness of these tools on an application of optimisation on manifolds in the setting of time series analysis. In this setting, orthogonal and unitary optimisation is used to constraint and regularise recurrent models and avoid vanishing and exploding gradient problems. The algorithms designed for GeoTorch allow us to achieve state of the art results in the standard tests for this family of models. We use tools from comparison geometry to give bounds on quantities that are of interest in optimisation problems. In particular, we build on the work of (Kaul 1976) to give explicit bounds on the norm of the second derivative of the Riemannian exponential.