DINOde: Continuous Vision-Text Alignment for Open-Vocabulary Semantic Segmentation
Not on the map. Why?
Abstract (arXiv)
Open-vocabulary semantic segmentation (OVSS) leverages textual semantics to segment objects beyond predefined categories. While the self-supervised model DINOv3 provides strong structured visual representations, its lack of native textual alignment hinders its direct application to OVSS. To bridge this gap, we propose DINOde, an ODE-based framework that continuously aligns CLIP text embeddings with the DINO visual manifold. Our approach employs two complementary components: (i) Semantic Text Flow (STF), which evolves text embeddings toward the DINO manifold through a continuous ODE trajectory, and (ii) Global Context Flow (GCF), which progressively refines the holistic image representation carried by DINO's CLS token. To preserve the hyperspherical geometry of the feature space during this evolution, we further introduce Velocity Tangent Projection, which constrains the learned velocity field to the tangent space. By modeling alignment as a continuous trajectory, DINOde avoids the manifold entanglement inherent in discrete MLP projections and yields more robust cross-modal alignment. Extensive experiments demonstrate that DINOde consistently outperforms existing methods and achieves state-of-the-art performance across multiple OVSS benchmarks. The code is available at https://github.com/yoon307/DINOde.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 70 of its 85 references that are in the map. First posted on arXiv 23 Jul 2026.
Builds on
70 of 85 references.
References · 70
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2023Segment Anythingcited by 8,158
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
38 more
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
- 2015Variational Inference with Normalizing Flowscited by 2,161
- 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
- 2025DINOv3cited by 1,297
- 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
- 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
- 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
- 2016Modeling Context in Referring Expressionscited by 1,054
- 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
- 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
- 2023Vision Transformers Need Registerscited by 844
- 2022Language-driven Semantic Segmentationcited by 711
- 2021Extract Free Dense Labels from CLIPcited by 580
- 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
- 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
- 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
- 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
- 2019Augmented Neural ODEscited by 396
- 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
- 2023Side Adapter Network for Open-Vocabulary Semantic Segmentationcited by 296
- 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
- 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 227
- 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Modelcited by 223
- 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 199
- 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
- 2023SCLIP: Rethinking Self-Attention for Dense Vision-Language Inferencecited by 169
- 2020Single-Stage Semantic Segmentation from Image Labelscited by 167
- 2022SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentationcited by 162
- 2020Riemannian Continuous Normalizing Flowscited by 143
- 2022Multi-class Token Transformer for Weakly Supervised Semantic Segmentationcited by 138
- 2022Open-Vocabulary Universal Image Segmentation with MaskCLIPcited by 131
- 2022Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairscited by 130
- 2022ReCo: Retrieve and Co-segment for Zero-shot Transfercited by 115
- 2022Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learningcited by 115
- 2024ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inferencecited by 112
- 2022TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cutcited by 111
- 2024ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentationcited by 110
- 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 104
50 most cited of 70 All 70 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.