mlsanityData to 7 Oct 2026

Why are hyperbolic neural networks effective? A study on hierarchical representation capability

arXiv 2402.02478 ↗First posted 4 Feb 2024cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Hyperbolic Neural Networks (HNNs), operating in hyperbolic space, have been widely applied in recent years, motivated by the existence of an optimal embedding in hyperbolic space that can preserve data hierarchical relationships (termed Hierarchical Representation Capability, HRC) more accurately than Euclidean space. However, there is no evidence to suggest that HNNs can achieve this theoretical optimal embedding, leading to much research being built on flawed motivations. In this paper, we propose a benchmark for evaluating HRC and conduct a comprehensive analysis of why HNNs are effective through large-scale experiments. Inspired by the analysis results, we propose several pre-training strategies to enhance HRC and improve the performance of downstream tasks, further validating the reliability of the analysis. Experiments show that HNNs cannot achieve the theoretical optimal embedding. The HRC is significantly affected by the optimization objectives and hierarchical structures, and enhancing HRC through pre-training strategies can significantly improve the performance of HNNs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 4 Feb 2024.

Builds on

25 of 40 references.

References · 25

  1. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 674
  2. 2019Hyperbolic Graph Convolutional Neural Networkscited by 400
  3. 2018Hyperbolic Neural Networkscited by 393
  4. 2018Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometrycited by 261
  5. 2019Hyperbolic Graph Neural Networkscited by 244
  6. 2019Hyperbolic Image Embeddingscited by 219
  7. 2018Representation Tradeoffs for Hyperbolic Embeddingscited by 204
  8. 2020Low-Dimensional Hyperbolic Knowledge Graph Embeddingscited by 195
  9. 2021Hyperbolic Deep Neural Networks: A Surveycited by 138
  10. 2019Continuous Hierarchical Representations with Poincar'e Variational Auto-Encoderscited by 111
  11. 2019Mixed-curvature Variational Autoencoderscited by 85
  12. 2019Hyperbolic Graph Attention Networkcited by 69
13 more
  1. 2021Probing BERT in Hyperbolic Spacescited by 50
  2. 2021Hyperbolic Variational Graph Neural Network for Modeling Dynamic Graphscited by 49
  3. 2022Hyperbolic Graph Neural Networks: A Review of Methods and Applicationscited by 40
  4. 2020Tree! I am no Tree! I am a Low Dimensional Hyperbolic Embeddingcited by 36
  5. 2020HyperText: Endowing FastText with Hyperbolic Geometrycited by 28
  6. 2022Hyperbolic Deep Reinforcement Learningcited by 25
  7. 2020Robust Large-Margin Learning in Hyperbolic Spacecited by 21
  8. 2022Dual Space Graph Contrastive Learningcited by 20
  9. 2022FisheyeHDK: Hyperbolic Deformable Kernel Learning for Ultra-Wide Field-of-View Image Recognitioncited by 10
  10. 2021Embedding Heterogeneous Networks into Hyperbolic Space Without Meta-pathcited by 9
  11. 2020Hyperbolic Distance Matricescited by 7
  12. 2022Hyperbolic Relevance Matching for Neural Keyphrase Extractioncited by 7
  13. 2021Hyperbolic Generative Adversarial Networkcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Minimax Rates for Hyperbolic Hierarchical Learningcited by 0