mlsanityData to 14 Oct 2026

Semantic Alignment in Hyperbolic Space for Open-Vocabulary Semantic Segmentation

arXiv 2605.08874 ↗First posted 9 May 2026cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Open-vocabulary semantic segmentation requires adapting image-level vision-language models such as CLIP to dense pixel-level prediction, which is challenging due to the mismatch between hierarchical structure and semantic alignment in the embedding space. While recent works leverage hyperbolic geometry to model hierarchical relationships, they align embeddings across hierarchical levels but overlook semantic misalignment among embeddings within the same level. In this work, we propose HyRo, a hyperbolic fine-tuning framework that decouples hierarchical and semantic alignment in the Poincar'e ball model. HyRo aligns hierarchical levels by adjusting the hyperbolic radius and refines semantic relationships through angular alignment using an orthogonal transformation that theoretically preserves the hyperbolic radius. Experiments on standard open-vocabulary semantic segmentation benchmarks demonstrate that HyRo achieves state-of-the-art performance over prior methods.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 33 of its 42 references that are in the map. First posted on arXiv 9 May 2026.

Builds on

33 of 42 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Compositional Entailment Learning for Hyperbolic Vision-Language Models9 Oct
  2. 2023Side Adapter Network for Open-Vocabulary Semantic Segmentation23 Feb
  3. 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model29 Dec

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  4. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  5. 2022Language-driven Semantic Segmentationcited by 717
  6. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
  7. 2021Extract Free Dense Labels from CLIPcited by 583
  8. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
  9. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 473
  10. 2018Hyperbolic Neural Networkscited by 402
  11. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  12. 2019Zero-Shot Semantic Segmentationcited by 245
18 more
  1. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
  2. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228
  3. 2019Hyperbolic Image Embeddingscited by 221
  4. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 219
  5. 2018Representation Tradeoffs for Hyperbolic Embeddingscited by 206
  6. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 201
  7. 2022ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentationcited by 178
  8. 2018Poincar'e GloVe: Hyperbolic Word Embeddingscited by 156
  9. 2023Hyperbolic Image-Text Representationscited by 136
  10. 2022Hyperbolic Vision Transformers: Combining Improvements in Metric Learningcited by 111
  11. 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 97
  12. 2018Embedding Text in Hyperbolic Spacescited by 76
  13. 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 62
  14. 2019Inferring Concept Hierarchies from Text Corpora via Hyperbolic Embeddingscited by 49
  15. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 35
  16. 2024Parameter-efficient Fine-tuning in Hyperspherical Space for Open-vocabulary Semantic Segmentationcited by 8
  17. 2024Learning Visual Hierarchies in Hyperbolic Space for Image Retrievalcited by 8
  18. 2025DPSeg: Dual-Prompt Cost Volume Learning for Open-Vocabulary Semantic Segmentationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Hierarchical Prompt Learning for Hyperbolic Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026HyperCLIP++: Fine-tuning CLIP forOpen-vocabulary Semantic Segmentation in Hyperbolic Space
  2. 2026Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models
  3. 2026Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation
  4. 2025FGAseg: Fine-Grained Pixel-Text Alignment for Open-Vocabulary Semantic Segmentation
  5. 2026dinov3.seg: Open-Vocabulary Semantic Segmentation with DINOv3
  6. 2026HyFL-CLIP: Hyperbolic Fine-Tuning of CLIP for Robust Long-Context Understanding
  7. 2025Modality Alignment across Trees on Heterogeneous Hyperbolic Manifolds
  8. 2025Efficient Redundancy Reduction for Open-Vocabulary Semantic Segmentation