mlsanityData to 14 Oct 2026

Global and Local Entailment Learning for Natural World Imagery

arXiv 2506.21476 ↗First posted 26 Jun 2025cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Learning the hierarchical structure of data in vision-language models is a significant challenge. Previous works have attempted to address this challenge by employing entailment learning. However, these approaches fail to model the transitive nature of entailment explicitly, which establishes the relationship between order and semantics within a representation space. In this work, we introduce Radial Cross-Modal Embeddings (RCME), a framework that enables the explicit modeling of transitivity-enforced entailment. Our proposed framework optimizes for the partial order of concepts within vision-language models. By leveraging our framework, we develop a hierarchical vision-language foundation model capable of representing the hierarchy in the Tree of Life. Our experiments on hierarchical species classification and hierarchical retrieval tasks demonstrate the enhanced performance of our models compared to the existing state-of-the-art models. Our code and models are open-sourced at https://vishu26.github.io/RCME/index.html.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 45 references that are in the map. First posted on arXiv 26 Jun 2025.

Builds on

37 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Fine-Grained Semantically Aligned Vision-Language Pre-Training4 Aug
  2. 2021FILIP: Fine-grained Interactive Language-Image Pre-Training9 Nov

Other references · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  3. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  4. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
  5. 2018Recognition in Terra Incognitacited by 658
  6. 2019Hyperbolic Graph Convolutional Neural Networkscited by 402
  7. 2018Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometrycited by 265
  8. 2020Low-Dimensional Hyperbolic Knowledge Graph Embeddingscited by 195
  9. 2023GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localizationcited by 187
  10. 2018Hyperbolic Entailment Cones for Learning Hierarchical Embeddingscited by 162
  11. 2023BioCLIP: A Vision Foundation Model for the Tree of Lifecited by 161
  12. 2018Poincar'e GloVe: Hyperbolic Word Embeddingscited by 156
23 more
  1. 2023Hyperbolic Image-Text Representationscited by 136
  2. 2015Order-Embeddings of Images and Languagecited by 113
  3. 2022Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understandingcited by 103
  4. 2018Embedding Text in Hyperbolic Spacescited by 76
  5. 2024Compositional Entailment Learning for Hyperbolic Vision-Language Modelscited by 74
  6. 2023PIGEON: Predicting Image Geolocationscited by 56
  7. 2019Inferring Concept Hierarchies from Text Corpora via Hyperbolic Embeddingscited by 49
  8. 2025Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversioncited by 43
  9. 2023MammalNet: A Large-scale Video Benchmark for Mammal Recognition and Behavior Understandingcited by 36
  10. 2023Revisiting Multimodal Representation in Contrastive Learning: From Patch and Token Embeddings to Finite Discrete Tokenscited by 35
  11. 2024TaxaBind: A Unified Embedding Space for Ecological Applicationscited by 35
  12. 2024INQUIRE: A Natural World Text-to-Image Retrieval Benchmarkcited by 33
  13. 2024BioTrove: A Large Curated Image Dataset Enabling AI for Biodiversitycited by 20
  14. 2024Emergent Visual-Semantic Hierarchies in Image-Text Representationscited by 19
  15. 2024CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scalecited by 18
  16. 2024Probabilistic Language-Image Pre-Trainingcited by 16
  17. 2024WildSAT: Learning Satellite Image Representations from Wildlife Observationscited by 13
  18. 2023BirdSAT: Cross-View Contrastive Masked Autoencoders for Bird Species Classification and Mappingcited by 12
  19. 2021Semi-Supervised Learning with Taxonomic Labelscited by 11
  20. 2024VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Imagescited by 10
  21. 2024Contrastive ground-level image and remote sensing pre-training improves representation learning for natural world imagerycited by 10
  22. 2023Shadow Cones: A Generalized Framework for Partial Order Embeddingscited by 3
  23. 2024MiTREE: Multi-input Transformer Ecoregion Encoder for Species Distribution Modellingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Modality Alignment across Trees on Heterogeneous Hyperbolic Manifoldscited by 3
  2. 2026Polaris: Coupled Orbital Polar Embeddings for Hierarchical Concept Learningcited by 1
  3. 2026Beyond Flat Labels: Level-Restricted Contrastive Learning for Hierarchical Fine-Grained Vision Classificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Explaining, Verifying, and Aligning Semantic Hierarchies in Vision-Language Model Embeddings
  2. 2026Cross-Modal Taxonomic Generalization in (Vision-) Language Models
  3. 2026ARGENT: Adaptive Hierarchical Image-Text Representations
  4. 2024Natural Language Inference Improves Compositionality in Vision-Language Models
  5. 2026Atomic Visual Entailment: Enhancing Zero-Shot Vision-Language Reasoning through Atomic Fact Decomposition and Learned Selection
  6. 20233VL: Using Trees to Improve Vision-Language Models' Interpretability
  7. 2026Learning Taxonomic Trees with Hierarchical Representation Regularization for Large Multimodal Models
  8. 2025Hierarchy-Aware Fine-Tuning of Vision-Language Models