DouC: Dual-Branch CLIP for Training-Free Open-Vocabulary Segmentation
Not on the map. Why?
Abstract (arXiv)
Open-vocabulary semantic segmentation requires assigning pixel-level semantic labels while supporting an open and unrestricted set of categories. Training-free CLIP-based approaches preserve strong zero-shot generalization but typically rely on a single inference mechanism, limiting their ability to jointly address unreliable local tokens and insufficient spatial coherence. We propose DouC, a training-free dual-branch CLIP framework that decomposes dense prediction into two complementary components. OG-CLIP improves patch-level reliability via lightweight, inference-time token gating, while FADE-CLIP injects external structural priors through proxy attention guided by frozen vision foundation models. The two branches are fused at the logit level, enabling local token reliability and structure-aware patch interactions to jointly influence final predictions, with optional instance-aware correction applied as post-processing. DouC introduces no additional learnable parameters, requires no retraining, and preserves CLIP's zero-shot generalization. Extensive experiments across eight benchmarks and multiple CLIP backbones demonstrate that DouC consistently outperforms prior training-free methods and scales favorably with model capacity.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 31 references that are in the map. First posted on arXiv 27 Apr 2026.
Builds on
27 of 31 references.
References · 27
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023Segment Anythingcited by 8,158
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
- 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
- 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
- 2023Demystifying CLIP Datacited by 265
- 2023Data Filtering Networkscited by 247
- 2023SCLIP: Rethinking Self-Attention for Dense Vision-Language Inferencecited by 169
15 more
- 2022Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairscited by 130
- 2024ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inferencecited by 112
- 2023SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understandingcited by 111
- 2024ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentationcited by 110
- 2023CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentationcited by 94
- 2024Pay Attention to Your Neighbours: Training-Free Open-Vocabulary Semantic Segmentationcited by 76
- 2025Your ViT is Secretly an Image Segmentation Modelcited by 61
- 2024Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generationcited by 43
- 2024Harnessing Vision Foundation Models for High-Performance, Training-Free Open Vocabulary Segmentationcited by 38
- 2024Self-Calibrated CLIP for Training-Free Open-Vocabulary Segmentationcited by 34
- 2024CorrCLIP: Reconstructing Patch Correlations in CLIP for Open-Vocabulary Semantic Segmentationcited by 33
- 2024ResCLIP: Residual Attention for Training-free Dense Vision-language Inferencecited by 28
- 2024Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentationcited by 23
- 2024CLIPer: Hierarchically Improving Spatial Representation of CLIP for Open-Vocabulary Semantic Segmentationcited by 17
- 2025Plug-in Feedback Self-adaptive Attention in CLIP for Training-free Open-Vocabulary Segmentationcited by 12
Built on by
Not cited yet.