# The Numerical Stability of Hyperbolic Representation Learning

arXiv: https://arxiv.org/abs/2211.00181 · HTML: https://mlsanity.com/paper/2211.00181/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 40 references that are in the map.
- First posted on arXiv 31 Oct 2022.

## Builds on (most cited first, 19 of 19)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry](https://mlsanity.com/paper/1806.03417/) (2018)
- [Hyperbolic Image Embeddings](https://mlsanity.com/paper/1904.02239/) (2019)
- [Representation Tradeoffs for Hyperbolic Embeddings](https://mlsanity.com/paper/1804.03329/) (2018)
- [Riemannian Adaptive Optimization Methods](https://mlsanity.com/paper/1810.00760/) (2018)
- [Hyperbolic Deep Neural Networks: A Survey](https://mlsanity.com/paper/2101.04562/) (2021)
- [Geoopt: Riemannian Optimization in PyTorch](https://mlsanity.com/paper/2005.02819/) (2020)
- [Continuous Hierarchical Representations with Poincar'e Variational Auto-Encoders](https://mlsanity.com/paper/1901.06033/) (2019)
- [Fully Hyperbolic Neural Networks](https://mlsanity.com/paper/2105.14686/) (2021)
- [Clipped Hyperbolic Classifiers Are Super-Hyperbolic Classifiers](https://mlsanity.com/paper/2107.11472/) (2021)
- [Large-Margin Classification in Hyperbolic Space](https://mlsanity.com/paper/1806.00437/) (2018)
- [HyperText: Endowing FastText with Hyperbolic Geometry](https://mlsanity.com/paper/2010.16143/) (2020)
- [A Fully Hyperbolic Neural Model for Hierarchical Multi-Class Classification](https://mlsanity.com/paper/2010.02053/) (2020)
- [Scalable Hyperbolic Recommender Systems](https://mlsanity.com/paper/1902.08648/) (2019)
- [Gradient descent in hyperbolic space](https://mlsanity.com/paper/1805.08207/) (2018)
- [Fine-Grained Entity Typing in Hyperbolic Space](https://mlsanity.com/paper/1906.02505/) (2019)
- [Highly Scalable and Provably Accurate Classification in Poincare Balls](https://mlsanity.com/paper/2109.03781/) (2021)

## Built on by (most cited first, 30 of 47)

- [Hyperbolic Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/2305.06611/) (2023)
- [Fully Hyperbolic Convolutional Neural Networks for Computer Vision](https://mlsanity.com/paper/2303.15919/) (2023)
- [Hypformer: Exploring Efficient Transformer Fully in Hyperbolic Space](https://mlsanity.com/paper/2407.01290/) (2024)
- [Hyperbolic Fine-Tuning for Large Language Models](https://mlsanity.com/paper/2410.04010/) (2024)
- [HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts](https://mlsanity.com/paper/2505.24722/) (2025)
- [Hyperbolic Diffusion Embedding and Distance for Hierarchical Representation Learning](https://mlsanity.com/paper/2305.18962/) (2023)
- [Learning Structured Representations with Hyperbolic Embeddings](https://mlsanity.com/paper/2412.01023/) (2024)
- [Generalizing Knowledge Graph Embedding with Universal Orthogonal Parameterization](https://mlsanity.com/paper/2405.08540/) (2024)
- [Mitigating Label Noise using Prompt-Based Hyperbolic Meta-Learning in Open-Set Domain Generalization](https://mlsanity.com/paper/2412.18342/) (2024)
- [Hierarchical Mamba Meets Hyperbolic Geometry: A New Paradigm for Structured Language Embeddings](https://mlsanity.com/paper/2505.18973/) (2025)
- [Hyperbolic Large Language Models](https://mlsanity.com/paper/2509.05757/) (2025)
- [Klein Model for Hyperbolic Neural Networks](https://mlsanity.com/paper/2410.16813/) (2024)
- [Is Hyperbolic Space All You Need for Medical Anomaly Detection?](https://mlsanity.com/paper/2505.21228/) (2025)
- [OpenHype: Hyperbolic Embeddings for Hierarchical Open-Vocabulary Radiance Fields](https://mlsanity.com/paper/2510.21441/) (2025)
- [Robust Hyperbolic Learning with Curvature-Aware Optimization](https://mlsanity.com/paper/2405.13979/) (2024)
- [Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learning](https://mlsanity.com/paper/2506.05826/) (2025)
- [Hyperbolic Metric Learning for Visual Outlier Detection](https://mlsanity.com/paper/2403.15260/) (2024)
- [MoE-Enhanced Explainable Deep Manifold Transformation for Complex Data Embedding and Visualization](https://mlsanity.com/paper/2410.19504/) (2024)
- [Fully Hyperbolic Rotation for Knowledge Graph Embedding](https://mlsanity.com/paper/2411.03622/) (2024)
- [Understanding and Improving Hyperbolic Deep Reinforcement Learning](https://mlsanity.com/paper/2512.14202/) (2025)
- [Fast and Geometrically Grounded Lorentz Neural Networks](https://mlsanity.com/paper/2601.21529/) (2026)
- [Intrinsic Lorentz Neural Network](https://mlsanity.com/paper/2602.23981/) (2026)
- [HyperbolicRAG: Enhancing Retrieval-Augmented Generation with Hyperbolic Representations](https://mlsanity.com/paper/2511.18808/) (2025)
- [HEEGNet: Hyperbolic Embeddings for EEG](https://mlsanity.com/paper/2601.03322/) (2026)
- [EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts](https://mlsanity.com/paper/2604.12579/) (2026)
- [Hyperbolic Multimodal Continual Learning](https://mlsanity.com/paper/2608.09572/) (2026)
- [Towards Non-Euclidean Foundation Models: Advancing AI Beyond Euclidean Frameworks](https://mlsanity.com/paper/2505.14417/) (2025)
- [HexFormer: Hyperbolic Vision Transformer with Exponential Map Aggregation](https://mlsanity.com/paper/2601.19849/) (2026)
- [hyperbolix: Hyperbolic Deep Learning in JAX](https://mlsanity.com/paper/2609.28248/) (2026)
- [Minkowski Attractor Networks: Closed-Form Hyperbolic Flows for Visual Representations](https://mlsanity.com/paper/2609.37817/) (2026)

## Abstract (arXiv)

Given the exponential growth of the volume of the ball w.r.t. its radius, the hyperbolic space is capable of embedding trees with arbitrarily small distortion and hence has received wide attention for representing hierarchical datasets. However, this exponential growth property comes at a price of numerical instability such that training hyperbolic learning models will sometimes lead to catastrophic NaN problems, encountering unrepresentable values in floating point arithmetic. In this work, we carefully analyze the limitation of two popular models for the hyperbolic space, namely, the Poincar'e ball and the Lorentz model. We first show that, under the 64 bit arithmetic system, the Poincar'e ball has a relatively larger capacity than the Lorentz model for correctly representing points. Then, we theoretically validate the superiority of the Lorentz model over the Poincar'e ball from the perspective of optimization. Given the numerical limitations of both models, we identify one Euclidean parametrization of the hyperbolic space which can alleviate these limitations. We further extend this Euclidean parametrization to hyperbolic hyperplanes and exhibits its ability in improving the performance of hyperbolic SVM.