mlsanityData to 7 Oct 2026

Hyperbolic Image-Text Representations

arXiv 2304.09172 ↗First posted 18 Apr 2023cs.CV cs.LGcited by 132

Not on the map. Why?

Cited by 26 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Visual and linguistic concepts naturally organize themselves in a hierarchy, where a textual concept "dog" entails all images that contain dogs. Despite being intuitive, current large-scale vision and language models such as CLIP do not explicitly capture such hierarchy. We propose MERU, a contrastive model that yields hyperbolic representations of images and text. Hyperbolic spaces have suitable geometric properties to embed tree-like data, so MERU can better capture the underlying hierarchy in image-text datasets. Our results show that MERU learns a highly interpretable and structured representation space while being competitive with CLIP's performance on standard multi-modal tasks like image classification and image-text retrieval. Our code and models are available at https://www.github.com/facebookresearch/meru

Key facts. Cited by 132 papers in the map (data through 7 Oct 2026). Builds on 60 of its 103 references that are in the map. First posted on arXiv 18 Apr 2023.

Builds on

60 of 103 references.

References · 60

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  10. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  11. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
38 more
  1. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2013Describing Textures in the Wildcited by 2,110
  4. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  5. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  6. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  7. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  8. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  9. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  10. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  11. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  12. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  13. 2017Mixed Precision Trainingcited by 1,135
  14. 2016Colorful Image Colorizationcited by 1,113
  15. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  16. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  17. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  18. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  19. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  20. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  21. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 674
  22. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  23. 2018Do Better ImageNet Models Transfer Better?cited by 560
  24. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  25. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
  26. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  27. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419
  28. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  29. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  30. 2022Matryoshka Representation Learningcited by 268
  31. 2018Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometrycited by 261
  32. 2019Hyperbolic Image Embeddingscited by 219
  33. 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 175
  34. 2018Hyperbolic Entailment Cones for Learning Hierarchical Embeddingscited by 160
  35. 2018Poincar'e GloVe: Hyperbolic Word Embeddingscited by 155
  36. 2020Learning Visual Representations with Caption Annotationscited by 123
  37. 2015Order-Embeddings of Images and Languagecited by 112
  38. 2022Hyperbolic Vision Transformers: Combining Improvements in Metric Learningcited by 109

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 33; 2025: 45; 2026: 49.

132 papers.

Sort:

By area:

  1. 2023Hyperbolic Deep Learning in Computer Vision: A Surveycited by 72
  2. 2024Compositional Entailment Learning for Hyperbolic Vision-Language Modelscited by 72
  3. 2024TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negativescited by 57
  4. 2024N2F2: Hierarchical Scene Understanding with Nested Neural Feature Fieldscited by 34
  5. 2024Hyperbolic Fine-Tuning for Large Language Modelscited by 24
  6. 2024A polar coordinate system represents syntax in large language modelscited by 24
  7. 2024MoDE: CLIP Data Experts via Clusteringcited by 22
  8. 2025HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Expertscited by 22
  9. 2024CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learningcited by 20
  10. 2025Is CLIP ideal? No. Can we fix it? Yes!cited by 20
  11. 2025Hyperbolic Deep Learning for Foundation Models: A Surveycited by 20
  12. 2024Emergent Visual-Semantic Hierarchies in Image-Text Representationscited by 19
38 more
  1. 2024HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Textscited by 17
  2. 2025Hyperbolic Safety-Aware Vision-Language Modelscited by 17
  3. 2024Probabilistic Language-Image Pre-Trainingcited by 16
  4. 2025Whitened CLIP as a Likelihood Surrogate of Images and Captionscited by 16
  5. 2025Position: Beyond Euclidean -- Foundation Models Should Embrace Non-Euclidean Geometriescited by 15
  6. 2025Hyperbolic Dataset Distillationcited by 15
  7. 2024HyperVQ: MLR-based Vector Quantization in Hyperbolic Spacecited by 13
  8. 2024Duoduo CLIP: Efficient 3D Understanding with Multi-View Imagescited by 12
  9. 2024Improving Visual Recognition with Hyperbolical Visual Hierarchy Mappingcited by 11
  10. 2024Hyperbolic Learning with Synthetic Captions for Open-World Detectioncited by 11
  11. 2024Hyperbolic Learning with Multimodal Large Language Modelscited by 10
  12. 2024Social Perception of Faces in a Vision-Language Modelcited by 10
  13. 2023Hyperbolic Random Forestscited by 9
  14. 2024HyperSDFusion: Bridging Hierarchical Structures in Language and Geometry for Enhanced 3D Text2Shape Generationcited by 8
  15. 2024Taxonomy-Aware Continual Semantic Segmentation in Hyperbolic Spaces for Open-World Perceptioncited by 8
  16. 2024Learning Visual Hierarchies in Hyperbolic Space for Image Retrievalcited by 8
  17. 2023Fast hyperboloid decision tree algorithmscited by 7
  18. 2024A Fashion Item Recommendation Model in Hyperbolic Spacecited by 7
  19. 2025The LLM Bottleneck: Why Open-Source Vision LLMs Struggle with Hierarchical Visual Recognitioncited by 7
  20. 2025PHyCLIP: ℓ1-Product of Hyperbolic Factors Unifies Hierarchy and Compositionality in Vision-Language Representation Learningcited by 7
  21. 2024Weighted Point Set Embedding for Multimodal Contrastive Learning Toward Optimal Similarity Metriccited by 6
  22. 2024Embedding Geometries of Contrastive Language-Image Pre-Trainingcited by 6
  23. 2025Low-distortion and GPU-compatible Tree Embeddings in Hyperbolic Spacecited by 6
  24. 2026GeoWorld: Geometric World Modelscited by 6
  25. 2024SPIN: Hierarchical Segmentation with Subpart Granularity in Natural Imagescited by 5
  26. 2025Is Hyperbolic Space All You Need for Medical Anomaly Detection?cited by 5
  27. 2025OpenHype: Hyperbolic Embeddings for Hierarchical Open-Vocabulary Radiance Fieldscited by 5
  28. 2024Product Manifold Representations for Learning on Biological Pathwayscited by 4
  29. 2024A streamlined Approach to Multimodal Few-Shot Class Incremental Learning for Fine-Grained Datasetscited by 4
  30. 2025Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learningcited by 4
  31. 2025Continual Hyperbolic Learning of Instances and Classescited by 4
  32. 2025On the rankability of visual embeddingscited by 4
  33. 2025HyperET: Efficient Training in Hyperbolic Space for Multi-modal Large Language Modelscited by 4
  34. 2026Path-Decoupled Hyperbolic Flow Matching for Few-Shot Adaptationcited by 4
  35. 2023Hypernymy Understanding Evaluation of Text-to-Image Models via WordNet Hierarchycited by 3
  36. 2024Klein Model for Hyperbolic Neural Networkscited by 3
  37. 2024HypDAE: Hyperbolic Diffusion Autoencoders for Hierarchical Few-shot Image Generationcited by 3
  38. 2024Adversarial Attacks on Hyperbolic Networkscited by 3

50 most cited of 132 All 132 (JSON)