mlsanityData to 7 Oct 2026

Hyperbolic Attention Networks

arXiv 1805.09786 ↗First posted 24 May 2018cs.NEcited by 122

Not on the map. Why?

Abstract (arXiv)

We introduce hyperbolic attention networks to endow neural networks with enough capacity to match the complexity of data with hierarchical and power-law structure. A few recent approaches have successfully demonstrated the benefits of imposing hyperbolic geometry on the parameters of shallow networks. We extend this line of work by imposing hyperbolic geometry on the activations of neural networks. This allows us to exploit hyperbolic geometry to reason about embeddings produced by deep networks. We achieve this by re-expressing the ubiquitous mechanism of soft attention in terms of operations defined for hyperboloid and Klein models. Our method shows improvements in terms of generalization on neural machine translation, learning on graphs and visual question answering tasks while keeping the neural representations compact.

Key facts. Cited by 122 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 19; 2023: 23; 2024: 15; 2025: 21; 2026: 16.

122 papers.

Sort:

By area:

  1. 2021Hyperbolic Deep Neural Networks: A Surveycited by 138
  2. 2021Fully Hyperbolic Neural Networkscited by 107
  3. 2023Hyperbolic Deep Learning in Computer Vision: A Surveycited by 72
  4. 2021Discrete-time Temporal Network Embedding via Implicit Hierarchical Learning in Hyperbolic Spacecited by 67
  5. 2021Lorentzian Graph Convolutional Networkscited by 66
  6. 2021Clipped Hyperbolic Classifiers Are Super-Hyperbolic Classifierscited by 56
  7. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  8. 2021Hyperbolic Variational Graph Neural Network for Modeling Dynamic Graphscited by 49
  9. 2021Learning the Predictability of the Futurecited by 48
  10. 2023HYperbolic Self-Paced Learning for Self-Supervised Skeleton-based Action Representationscited by 42
  11. 2022Hyperbolic Graph Neural Networks: A Review of Methods and Applicationscited by 40
  12. 2021A Self-supervised Mixed-curvature Graph Neural Networkcited by 32
38 more
  1. 2022Self-Supervised Continual Graph Learning in Adaptive Riemannian Spacescited by 32
  2. 2021CO-SNE: Dimensionality Reduction and Visualization for Hyperbolic Datacited by 28
  3. 2021Learning Conjoint Attentions for Graph Neural Netscited by 26
  4. 2022A Self-supervised Riemannian GNN with Time Varying Curvature for Temporal Graph Learningcited by 24
  5. 2024Hyperbolic Fine-Tuning for Large Language Modelscited by 24
  6. 2022Heterogeneous manifolds for curvature-aware graph embeddingcited by 23
  7. 2025HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Expertscited by 22
  8. 2023Hyperbolic Representation Learning: Revisiting and Advancingcited by 21
  9. 2023Riemannian Residual Neural Networkscited by 21
  10. 2024Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structurescited by 20
  11. 2025Hyperbolic Category Discoverycited by 20
  12. 2025Hyperbolic Deep Learning for Foundation Models: A Surveycited by 20
  13. 2021Multi-modal Entity Alignment in Hyperbolic Spacecited by 19
  14. 2023Hyperbolic Audio-visual Zero-shot Learningcited by 19
  15. 2023Hyperbolic Active Learning for Semantic Segmentation under Domain Shiftcited by 18
  16. 2024Learning Structured Representations with Hyperbolic Embeddingscited by 16
  17. 2021Unsupervised Hyperbolic Representation Learning via Message Passing Auto-Encoderscited by 15
  18. 2023Curve Your Attention: Mixed-Curvature Transformers for Graph Representation Learningcited by 15
  19. 2025Position: Beyond Euclidean -- Foundation Models Should Embrace Non-Euclidean Geometriescited by 15
  20. 2021HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learningcited by 14
  21. 2021Nested Hyperbolic Spaces for Dimensionality Reduction and Hyperbolic NN Designcited by 14
  22. 2022Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysiscited by 14
  23. 2021Extracting Event Temporal Relations via Hyperbolic Geometrycited by 13
  24. 2022Are we certain it's anomalous?cited by 13
  25. 2021Knowledge-Enhanced Top-K Recommendation in Poincar'e Ballcited by 12
  26. 2021Generalization Error Bound for Hyperbolic Ordinal Embeddingcited by 12
  27. 2023Learning Weakly Supervised Audio-Visual Violence Detection in Hyperbolic Spacecited by 12
  28. 2023Leveraging Label Non-Uniformity for Node Classification in Graph Neural Networkscited by 11
  29. 2023Dynamic Hyperbolic Attention Network for Fine Hand-object Reconstructioncited by 10
  30. 2024Hyperbolic Learning with Multimodal Large Language Modelscited by 10
  31. 2022Hyperbolic Hierarchical Contrastive Hashingcited by 9
  32. 2022Random Laplacian Features for Learning with Hyperbolic Spacecited by 8
  33. 2024HVT: A Comprehensive Vision Framework for Learning in Non-Euclidean Spacecited by 8
  34. 2021Learning Hyperbolic Representations of Topological Featurescited by 7
  35. 2022Hyperbolic Relevance Matching for Neural Keyphrase Extractioncited by 7
  36. 2023Fast hyperboloid decision tree algorithmscited by 7
  37. 2024A Fashion Item Recommendation Model in Hyperbolic Spacecited by 7
  38. 2021Self-supervised Video Representation Learning with Cross-Stream Prototypical Contrastingcited by 6

50 most cited of 122 All 122 (JSON)