# Why are hyperbolic neural networks effective? A study on hierarchical representation capability

arXiv: https://arxiv.org/abs/2402.02478 · HTML: https://mlsanity.com/paper/2402.02478/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 4 Feb 2024.

## Builds on (most cited first, 25 of 25)

- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [Hyperbolic Graph Convolutional Neural Networks](https://mlsanity.com/paper/1910.12933/) (2019)
- [Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry](https://mlsanity.com/paper/1806.03417/) (2018)
- [Hyperbolic Graph Neural Networks](https://mlsanity.com/paper/1910.12892/) (2019)
- [Hyperbolic Image Embeddings](https://mlsanity.com/paper/1904.02239/) (2019)
- [Representation Tradeoffs for Hyperbolic Embeddings](https://mlsanity.com/paper/1804.03329/) (2018)
- [Low-Dimensional Hyperbolic Knowledge Graph Embeddings](https://mlsanity.com/paper/2005.00545/) (2020)
- [Hyperbolic Deep Neural Networks: A Survey](https://mlsanity.com/paper/2101.04562/) (2021)
- [Continuous Hierarchical Representations with Poincar'e Variational Auto-Encoders](https://mlsanity.com/paper/1901.06033/) (2019)
- [Mixed-curvature Variational Autoencoders](https://mlsanity.com/paper/1911.08411/) (2019)
- [Hyperbolic Graph Attention Network](https://mlsanity.com/paper/1912.03046/) (2019)
- [Hyperbolic Variational Graph Neural Network for Modeling Dynamic Graphs](https://mlsanity.com/paper/2104.02228/) (2021)
- [Probing BERT in Hyperbolic Spaces](https://mlsanity.com/paper/2104.03869/) (2021)
- [Hyperbolic Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/2202.13852/) (2022)
- [Tree! I am no Tree! I am a Low Dimensional Hyperbolic Embedding](https://mlsanity.com/paper/2005.03847/) (2020)
- [HyperText: Endowing FastText with Hyperbolic Geometry](https://mlsanity.com/paper/2010.16143/) (2020)
- [Hyperbolic Deep Reinforcement Learning](https://mlsanity.com/paper/2210.01542/) (2022)
- [Robust Large-Margin Learning in Hyperbolic Space](https://mlsanity.com/paper/2004.05465/) (2020)
- [Dual Space Graph Contrastive Learning](https://mlsanity.com/paper/2201.07409/) (2022)
- [FisheyeHDK: Hyperbolic Deformable Kernel Learning for Ultra-Wide Field-of-View Image Recognition](https://mlsanity.com/paper/2203.07255/) (2022)
- [Embedding Heterogeneous Networks into Hyperbolic Space Without Meta-path](https://mlsanity.com/paper/2106.09923/) (2021)
- [Hyperbolic Distance Matrices](https://mlsanity.com/paper/2005.08672/) (2020)
- [Hyperbolic Relevance Matching for Neural Keyphrase Extraction](https://mlsanity.com/paper/2205.02047/) (2022)
- [Hyperbolic Generative Adversarial Network](https://mlsanity.com/paper/2102.05567/) (2021)

## Built on by (most cited first, 1 of 1)

- [Minimax Rates for Hyperbolic Hierarchical Learning](https://mlsanity.com/paper/2601.20047/) (2026)

## Abstract (arXiv)

Hyperbolic Neural Networks (HNNs), operating in hyperbolic space, have been widely applied in recent years, motivated by the existence of an optimal embedding in hyperbolic space that can preserve data hierarchical relationships (termed Hierarchical Representation Capability, HRC) more accurately than Euclidean space. However, there is no evidence to suggest that HNNs can achieve this theoretical optimal embedding, leading to much research being built on flawed motivations. In this paper, we propose a benchmark for evaluating HRC and conduct a comprehensive analysis of why HNNs are effective through large-scale experiments. Inspired by the analysis results, we propose several pre-training strategies to enhance HRC and improve the performance of downstream tasks, further validating the reliability of the analysis. Experiments show that HNNs cannot achieve the theoretical optimal embedding. The HRC is significantly affected by the optimization objectives and hierarchical structures, and enhancing HRC through pre-training strategies can significantly improve the performance of HNNs.