# Hyperbolic Optimization

arXiv: https://arxiv.org/abs/2509.25206 · HTML: https://mlsanity.com/paper/2509.25206/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 20 Sep 2025.

## Builds on (most cited first, 12 of 12)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Fully Hyperbolic Neural Networks](https://mlsanity.com/paper/2105.14686/) (2021)
- [Hyperbolic Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/2305.06611/) (2023)
- [Quasi-hyperbolic momentum and Adam for deep learning](https://mlsanity.com/paper/1810.06801/) (2018)
- [Canonical normalizing flows for manifold learning](https://mlsanity.com/paper/2310.12743/) (2023)
- [In-Context Learning State Vector with Inner and Momentum Optimization](https://mlsanity.com/paper/2404.11225/) (2024)
- [A Matrix Product State Model for Simultaneous Classification and Generation](https://mlsanity.com/paper/2406.17441/) (2024)
- [Continual Hyperbolic Learning of Instances and Classes](https://mlsanity.com/paper/2506.10710/) (2025)
- [A Survey of Optimization Methods for Training DL Models: Theoretical Perspective on Convergence and Generalization](https://mlsanity.com/paper/2501.14458/) (2025)
- [Learning Acceleration Algorithms for Fast Parametric Convex Optimization with Certified Robustness](https://mlsanity.com/paper/2507.16264/) (2025)

## Abstract (arXiv)

This work explores optimization methods on hyperbolic manifolds. Building on Riemannian optimization principles, we extend the Hyperbolic Stochastic Gradient Descent (a specialization of Riemannian SGD) to a Hyperbolic Adam optimizer. While these methods are particularly relevant for learning on the Poincar'e ball, they may also provide benefits in Euclidean and other non-Euclidean settings, as the chosen optimization encourages the learning of Poincar'e embeddings. This representation, in turn, accelerates convergence in the early stages of training, when parameters are far from the optimum. As a case study, we train diffusion models using the hyperbolic optimization methods with hyperbolic time-discretization of the Langevin dynamics, and show that they achieve faster convergence on certain datasets without sacrificing generative quality.