mlsanityData to 7 Oct 2026

Hyperbolic Deep Learning for Foundation Models: A Survey

arXiv 2507.17787 ↗First posted 23 Jul 2025cs.LG cs.AIcited by 20

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Foundation models pre-trained on massive datasets, including large language models (LLMs), vision-language models (VLMs), and large multimodal models, have demonstrated remarkable success in diverse downstream tasks. However, recent studies have shown fundamental limitations of these models: (1) limited representational capacity, (2) lower adaptability, and (3) diminishing scalability. These shortcomings raise a critical question: is Euclidean geometry truly the optimal inductive bias for all foundation models, or could incorporating alternative geometric spaces enable models to better align with the intrinsic structure of real-world data and improve reasoning processes? Hyperbolic spaces, a class of non-Euclidean manifolds characterized by exponential volume growth with respect to distance, offer a mathematically grounded solution. These spaces enable low-distortion embeddings of hierarchical structures (e.g., trees, taxonomies) and power-law distributions with substantially fewer dimensions compared to Euclidean counterparts. Recent advances have leveraged these properties to enhance foundation models, including improving LLMs' complex reasoning ability, VLMs' zero-shot generalization, and cross-modal semantic alignment, while maintaining parameter efficiency. This paper provides a comprehensive review of hyperbolic neural networks and their recent development for foundation models. We further outline key challenges and research directions to advance the field.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 66 of its 110 references that are in the map. First posted on arXiv 23 Jul 2025.

Builds on

66 of 110 references.

References · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  8. 2016Layer Normalizationcited by 5,579
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  12. 2022Training Compute-Optimal Large Language Modelscited by 2,927
38 more
  1. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  4. 2022Diffusion Models in Vision: A Surveycited by 1,008
  5. 2019An Introduction to Variational Autoencoderscited by 997
  6. 2020Simple and Deep Graph Convolutional Networkscited by 857
  7. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  8. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 674
  9. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  10. 2015An Introduction to Convolutional Neural Networkscited by 510
  11. 2019Hyperbolic Graph Convolutional Neural Networkscited by 400
  12. 2018Hyperbolic Neural Networkscited by 393
  13. 2018Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometrycited by 261
  14. 2019Hyperbolic Graph Neural Networkscited by 244
  15. 2019Visualizing and Measuring the Geometry of BERTcited by 215
  16. 2018Representation Tradeoffs for Hyperbolic Embeddingscited by 204
  17. 2022Convergence for score-based generative modeling with polynomial complexitycited by 176
  18. 2018Hyperbolic Entailment Cones for Learning Hierarchical Embeddingscited by 160
  19. 2018Poincar'e GloVe: Hyperbolic Word Embeddingscited by 155
  20. 2020Adversarial Self-Supervised Contrastive Learningcited by 138
  21. 2021Hyperbolic Deep Neural Networks: A Surveycited by 138
  22. 2024The Revolution of Multimodal Large Language Models: A Surveycited by 134
  23. 2023Hyperbolic Image-Text Representationscited by 132
  24. 2018Hyperbolic Attention Networkscited by 122
  25. 2019Continuous Hierarchical Representations with Poincar'e Variational Auto-Encoderscited by 111
  26. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 111
  27. 2020Geoopt: Riemannian Optimization in PyTorchcited by 109
  28. 2022Hyperbolic Vision Transformers: Combining Improvements in Metric Learningcited by 109
  29. 2021Fully Hyperbolic Neural Networkscited by 107
  30. 2021Unsupervised Object-Level Representation Learning from Scene Imagescited by 72
  31. 2018geomstats: a Python Package for Riemannian Geometry in Machine Learningcited by 71
  32. 2019Hyperbolic Graph Attention Networkcited by 69
  33. 2021Discrete-time Temporal Network Embedding via Implicit Hierarchical Learning in Hyperbolic Spacecited by 67
  34. 2021Lorentzian Graph Convolutional Networkscited by 66
  35. 2022Hyperbolic Contrastive Learning for Visual Representations beyond Objectscited by 58
  36. 2020Graph Geometry Interaction Learningcited by 55
  37. 2021A Hyperbolic-to-Hyperbolic Graph Convolutional Networkcited by 53
  38. 2020Differentiating through the Fr'echet Meancited by 52

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 16.

20 papers.

Sort:

By area:

  1. 2025Angular Gradient Sign Method: Uncovering Vulnerabilities in Hyperbolic Networkscited by 2
  2. 2025HyperbolicRAG: Enhancing Retrieval-Augmented Generation with Hyperbolic Representationscited by 2
  3. 2026Fast and Geometrically Grounded Lorentz Neural Networkscited by 2
  4. 2026Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Modelscited by 2
  5. 2025Randomized HyperSteiner: A Stochastic Delaunay Triangulation Heuristic for the Hyperbolic Steiner Minimal Treecited by 1
  6. 2026HypRAG: Hyperbolic Dense Retrieval for Retrieval Augmented Generationcited by 1
  7. 2026A Neural Tension Operator for Curve Subdivision across Constant Curvature Geometriescited by 1
  8. 2026HypEHR: Hyperbolic Modeling of Electronic Health Records for Efficient Question Answeringcited by 1
  9. 2026The Gate Always Closes: On Injecting Auxiliary Signals into Frozen Vision-Language Modelscited by 1
  10. 2026Riemann GeoResolver: A Non-Euclidean Attention Framework from Euclidean Resolver to Hyperbolic-Spherical Geometrycited by 1
  11. 2025More Than A Shortcut: A Hyperbolic Approach To Early-Exit Networkscited by 0
  12. 2026Tutorial on Reasoning for IR & IR for Reasoningcited by 0
8 more
  1. 2026HYDRA: Hyperbolic Dynamic Representation Architecture for Kolmogorov-Arnold Networkscited by 0
  2. 2026Adaptive Interpolatory Curve Subdivision with Learned Local Anglescited by 0
  3. 2026Hierarchical Prompt Learning for Hyperbolic Vision-Language Modelscited by 0
  4. 2026Geometry-Aware Hyperbolic Residual-Quantized Variational Autoencoderscited by 0
  5. 2026hyperbolix: Hyperbolic Deep Learning in JAXcited by 0
  6. 2026Rethinking the Fully Hyperbolic Vision Transformer in Polar Coordinatescited by 0
  7. 2026Explaining Hyperbolic Neural Networks via Geometry-Aware Relevance Propagationcited by 0
  8. 2026CurvSpec: Adaptive Multi-Curvature Learning for Partial Relevant Video Retrievalcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Hyperbolic Large Language Models
  2. 2025HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts
  3. 2024Hyperbolic Learning with Multimodal Large Language Models
  4. 2024Hyperbolic Fine-Tuning for Large Language Models
  5. 2025Training-Free Dual Hyperbolic Adapters for Better Cross-Modal Reasoning
  6. 2026Is the Geometry Doing the Work? An Operating-Point Audit of Hierarchy in Hyperbolic Vision-Language Models
  7. 2025Hierarchical Mamba Meets Hyperbolic Geometry: A New Paradigm for Structured Language Embeddings
  8. 2022Hierarchical Symbolic Reasoning in Hyperbolic Space for Deep Discriminative Models