# Hyperbolic Deep Neural Networks: A Survey

arXiv: https://arxiv.org/abs/2101.04562 · HTML: https://mlsanity.com/paper/2101.04562/

## Key facts

- Cited by 139 papers in the map (data through 14 Oct 2026).
- Builds on 100 of its 215 references that are in the map.
- First posted on arXiv 12 Jan 2021.

## Builds on (most cited first, 30 of 100)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)

## Built on by (most cited first, 30 of 139)

- [Hyperbolic Image Segmentation](https://mlsanity.com/paper/2203.05898/) (2022)
- [Compositional Entailment Learning for Hyperbolic Vision-Language Models](https://mlsanity.com/paper/2410.06912/) (2024)
- [Hyperbolic Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/2305.06611/) (2023)
- [E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning](https://mlsanity.com/paper/2307.13770/) (2023)
- [Fully Hyperbolic Convolutional Neural Networks for Computer Vision](https://mlsanity.com/paper/2303.15919/) (2023)
- [The Numerical Stability of Hyperbolic Representation Learning](https://mlsanity.com/paper/2211.00181/) (2022)
- [Knowledge Graph Embedding: A Survey from the Perspective of Representation Spaces](https://mlsanity.com/paper/2211.03536/) (2022)
- [Hyperbolic Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/2202.13852/) (2022)
- [The Exact Sample Complexity Gain from Invariances for Kernel Regression](https://mlsanity.com/paper/2303.14269/) (2023)
- [Hyperbolic Fine-Tuning for Large Language Models](https://mlsanity.com/paper/2410.04010/) (2024)
- [Hyperbolic Deep Reinforcement Learning](https://mlsanity.com/paper/2210.01542/) (2022)
- [Poincar'e ResNet](https://mlsanity.com/paper/2303.14027/) (2023)
- [Graph Neural Networks Designed for Different Graph Types: A Survey](https://mlsanity.com/paper/2204.03080/) (2022)
- [Hyperbolic Representation Learning: Revisiting and Advancing](https://mlsanity.com/paper/2306.09118/) (2023)
- [Hyperbolic Deep Learning for Foundation Models: A Survey](https://mlsanity.com/paper/2507.17787/) (2025)
- [Hyperbolic Audio-visual Zero-shot Learning](https://mlsanity.com/paper/2308.12558/) (2023)
- [Hardness of Learning Neural Networks under the Manifold Hypothesis](https://mlsanity.com/paper/2406.01461/) (2024)
- [Neural Distance Embeddings for Biological Sequences](https://mlsanity.com/paper/2109.09740/) (2021)
- [Hyperbolic Uncertainty Aware Semantic Segmentation](https://mlsanity.com/paper/2203.08881/) (2022)
- [Lorentzian Residual Neural Networks](https://mlsanity.com/paper/2412.14695/) (2024)
- [Hyperbolic Safety-Aware Vision-Language Models](https://mlsanity.com/paper/2503.12127/) (2025)
- [Hyperbolic Sliced-Wasserstein via Geodesic and Horospherical Projections](https://mlsanity.com/paper/2211.10066/) (2022)
- [Hyperbolic Contrastive Learning](https://mlsanity.com/paper/2302.01409/) (2023)
- [HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learning](https://mlsanity.com/paper/2109.10500/) (2021)
- [kHGCN: Tree-likeness Modeling via Continuous and Discrete Curvature Learning](https://mlsanity.com/paper/2212.01793/) (2022)
- [HGWaveNet: A Hyperbolic Graph Neural Network for Temporal Link Prediction](https://mlsanity.com/paper/2304.07302/) (2023)
- [HyperVQ: MLR-based Vector Quantization in Hyperbolic Space](https://mlsanity.com/paper/2403.13015/) (2024)
- [Generalization Error Bound for Hyperbolic Ordinal Embedding](https://mlsanity.com/paper/2105.10475/) (2021)
- [Rethinking Few-Shot Class-Incremental Learning with Open-Set Hypothesis in Hyperbolic Geometry](https://mlsanity.com/paper/2207.09963/) (2022)
- [Learning Weakly Supervised Audio-Visual Violence Detection in Hyperbolic Space](https://mlsanity.com/paper/2305.18797/) (2023)

## Abstract (arXiv)

Recently, there has been a rising surge of momentum for deep representation learning in hyperbolic spaces due to theirhigh capacity of modeling data like knowledge graphs or synonym hierarchies, possessing hierarchical structure. We refer to the model as hyperbolic deep neural network in this paper. Such a hyperbolic neural architecture potentially leads to drastically compact model withmuch more physical interpretability than its counterpart in Euclidean space. To stimulate future research, this paper presents acoherent and comprehensive review of the literature around the neural components in the construction of hyperbolic deep neuralnetworks, as well as the generalization of the leading deep approaches to the Hyperbolic space. It also presents current applicationsaround various machine learning tasks on several publicly available datasets, together with insightful observations and identifying openquestions and promising future directions.