mlsanityData to 14 Oct 2026

Order-Embeddings of Images and Language

arXiv 1511.06361 ↗First posted 19 Nov 2015cs.LG cs.CL cs.CVcited by 113

Not on the map. Why?

Abstract (arXiv)

Hypernymy, textual entailment, and image captioning can be seen as special cases of a single visual-semantic hierarchy over words, sentences, and images. In this paper we advocate for explicitly modeling the partial order structure of this hierarchy. Towards this goal, we introduce a general method for learning ordered representations, and show how it can be applied to a variety of tasks involving images and language. We show that the resulting representations improve performance over current approaches for hypernym prediction and image-caption retrieval.

Key facts. Cited by 113 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 20; 2022: 22; 2023: 11; 2024: 18; 2025: 20; 2026: 22.

113 papers.

Sort:

By area:

  1. 2021Hyperbolic Deep Neural Networks: A Surveycited by 139
  2. 2023Hyperbolic Image-Text Representationscited by 136
  3. 2021Similarity Reasoning and Filtration for Image-Text Matchingcited by 121
  4. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  5. 2022Deep Hierarchical Semantic Segmentationcited by 84
  6. 2021ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphscited by 78
  7. 2024Compositional Entailment Learning for Hyperbolic Vision-Language Modelscited by 74
  8. 2021DOC2PPT: Automatic Presentation Slides Generation from Scientific Documentscited by 70
  9. 20233D Concept Learning and Reasoning from Multi-View Imagescited by 60
  10. 2021Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Modelscited by 41
  11. 2021Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Conescited by 38
  12. 2021Modeling Fine-Grained Entity Types with Box Embeddingscited by 36
38 more
  1. 2022Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrievalcited by 34
  2. 2022TaxoCom: Topic Taxonomy Completion with Hierarchical Discovery of Novel Topic Clusterscited by 30
  3. 2022Improved Representation of Asymmetrical Distances with Interval Quasimetric Embeddingscited by 24
  4. 2021Structured Multi-modal Feature Embedding and Alignment for Image-Sentence Retrievalcited by 23
  5. 2021Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoningcited by 22
  6. 2022FALCON: Fast Visual Concept Learning by Integrating Images, Linguistic descriptions, and Conceptual Relationscited by 20
  7. 2021COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modalitycited by 19
  8. 2024Emergent Visual-Semantic Hierarchies in Image-Text Representationscited by 19
  9. 2024HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Textscited by 17
  10. 2025Hyperbolic Safety-Aware Vision-Language Modelscited by 17
  11. 2022On the Learning and Learnability of Quasimetricscited by 15
  12. 2022Cross-modal Semantic Enhanced Interaction for Image-Sentence Retrievalcited by 14
  13. 2021Extracting Event Temporal Relations via Hyperbolic Geometrycited by 13
  14. 20223D Concept Grounding on Neural Fieldscited by 13
  15. 2022FETA: Towards Specializing Foundation Models for Expert Task Applicationscited by 13
  16. 2024Ontology Embedding: A Survey of Methods, Applications and Resourcescited by 12
  17. 2024Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understandingcited by 12
  18. 2021Word2Box: Capturing Set-Theoretic Semantics of Words using Box Embeddingscited by 11
  19. 2024Resolving Word Vagueness with Scenario-guided Adapter for Natural Language Inferencecited by 11
  20. 2021Directed Graph Embeddings in Pseudo-Riemannian Manifoldscited by 9
  21. 2021Box Embeddings: An open-source library for representation learning using geometric structurescited by 9
  22. 2021TopicNet: Semantic Graph-Guided Topic Discoverycited by 9
  23. 2023Geometric Relational Embeddings: A Surveycited by 9
  24. 2023Towards Fast and Accurate Image-Text Retrieval with Self-Supervised Fine-Grained Alignmentcited by 8
  25. 2025Asymmetric Visual Semantic Embedding Framework for Efficient Vision-Language Alignmentcited by 8
  26. 2021Cross-Modal Generative Augmentation for Visual Question Answeringcited by 7
  27. 2022A Survey on Visual Transfer Learning using Knowledge Graphscited by 7
  28. 2024Hierarchical Indexing for Retrieval-Augmented Opinion Summarizationcited by 7
  29. 2025PHyCLIP: ℓ1-Product of Hyperbolic Factors Unifies Hierarchy and Compositionality in Vision-Language Representation Learningcited by 7
  30. 2024Representation Learning for Frequent Subgraph Miningcited by 5
  31. 2022Brainish: Formalizing A Multimodal Language for Intelligence and Consciousnesscited by 4
  32. 2025Decentralizing AI Memory: SHIMI, a Semantic Hierarchical Memory Index for Scalable Agent Reasoningcited by 4
  33. 2025On the rankability of visual embeddingscited by 4
  34. 2025Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipecited by 4
  35. 2022Word Embedding for Social Sciences: An Interdisciplinary Surveycited by 3
  36. 2022On the Complexity of Bayesian Generalizationcited by 3
  37. 2023Multivariate Representation Learning for Information Retrievalcited by 3
  38. 2023Shadow Cones: A Generalized Framework for Partial Order Embeddingscited by 3

50 most cited of 113 All 113 (JSON)