Is the Geometry Doing the Work? An Operating-Point Audit of Hierarchy in Hyperbolic Vision-Language Models
On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP
Abstract (arXiv)
Hyperbolic vision-language models are designed to encode abstraction geometrically: general concepts near the origin, specific ones farther out, and entailment cones representing directed order. We ask whether trained MERU, HyCoCLIP, and PHyCLIP models actually use these mechanisms. We audit seven released checkpoints and matched from-scratch interventions, using diagnostics that distinguish active hyperbolic geometry from angular structure and supervision effects. All audited converged checkpoints remain near-Euclidean in the dimensionless radius u=√cρ, which measures how strongly embeddings experience hyperbolic geometry: the largest observed image-side value is 0.37 -- well below u≈0.84, where local metric distortion reaches 10%. Releasing the curvature floor changes curvature and norms but not this regime, with mixed, generally modest downstream shifts. Trained entailment cones are saturated or nearly saturated, so low violation rates can arise from trivially wide cones rather than learned order. Preregistered semantic traversal detects weak within-branch order but no operative full-hierarchy readout. Shuffle-controlled tests detect no pair-specific radial ordering in released checkpoints, and no positive result is consistent across all three matched ViT-B seeds. We trace this to a low-curvature shortcut: lowering curvature widens entailment cones and suppresses violations without learning order. In the probed trajectories, gradient decomposition identifies entailment as the dominant curvature-lowering pressure during collapse. Yet curvature contracts even when entailment is removed, so the shortcut is not the sole cause. Under our diagnostics, the audited formulations do not demonstrate an operative radial or cone-based hierarchy. We distill the audit into a five-number geometry report for evaluating future hierarchy claims.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 28 references that are in the map. First posted on arXiv 6 Jul 2026.
Builds on
18 of 28 references.
Closest ancestors · 4
Same direction, newest first
- 2026ARGENT: Adaptive Hierarchical Image-Text Representations24 Mar
- 2024Compositional Entailment Learning for Hyperbolic Vision-Language Models9 Oct
- 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality26 Jun
- 2023HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attention6 Mar
Other references · 14
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
- 2014Microsoft COCO: Common Objects in Contextcited by 17,550
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,476
- 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,082
- 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 678
- 2018Representation Tradeoffs for Hyperbolic Embeddingscited by 205
- 2018Hyperbolic Entailment Cones for Learning Hierarchical Embeddingscited by 160
- 2023Hyperbolic Image-Text Representationscited by 133
- 2022VL-CheckList: Evaluating Pre-trained Vision-Language Models with Objects, Attributes and Relationscited by 114
- 2015Order-Embeddings of Images and Languagecited by 112
- 2021Clipped Hyperbolic Classifiers Are Super-Hyperbolic Classifierscited by 56
- 2025PHyCLIP: ℓ1-Product of Hyperbolic Factors Unifies Hierarchy and Compositionality in Vision-Language Representation Learningcited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Hierarchical Prompt Learning for Hyperbolic Vision-Language Models
- 2026Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models
- 2026Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects?
- 2026Separating perception from reasoning in vision-language models: a model-free render ceiling for crystal structures
- 2026Why Far Looks Up: Probing Spatial Representation in Vision-Language Models
- 2025Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation
- 2026HyperVis: Continuous Latent Visual Relational Graphs on the Lorentz Hyperboloid for Compositional Reasoning
- 2026Consistent Yet Wrong: Evidence Insensitivity in Spatial Vision-Language Models