PEARL: Geometry Aligns Semantics for Training-Free Open-Vocabulary Semantic Segmentation
Not on the map. Why?
Abstract (arXiv)
Training-free open-vocabulary semantic segmentation (OVSS) promises rapid adaptation to new label sets without retraining. Yet, many methods rely on heavy post-processing or handle text and vision in isolation, leaving cross-modal geometry underutilized. Others introduce auxiliary vision backbones or multi-model pipelines, which increase complexity and latency while compromising design simplicity. We present PEARL, textbfunderlineProcrusttextbfunderlinees textbfunderlinealignment with text-awatextbfunderlinere textbfunderlineLaplacian propagation, a compact two-step inference that follows an align-then-propagate principle. The Procrustes alignment step performs an orthogonal projection inside the last self-attention block, rotating keys toward the query subspace via a stable polar iteration. The text-aware Laplacian propagation then refines per-pixel logits on a small grid through a confidence-weighted, text-guided graph solve: text provides both a data-trust signal and neighbor gating, while image gradients preserve boundaries. In this work, our method is fully training-free, plug-and-play, and uses only fixed constants, adding minimal latency with a small per-head projection and a few conjugate-gradient steps. Our approach, PEARL, sets a new state-of-the-art in training-free OVSS without extra data or auxiliary backbones across standard benchmarks, achieving superior performance under both with-background and without-background protocols.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 71 of its 82 references that are in the map. First posted on arXiv 23 Mar 2026.
Builds on
71 of 82 references.
References · 71
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2023Segment Anythingcited by 8,158
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
38 more
- 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
- 2025DINOv3cited by 1,297
- 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
- 2021Grounded Language-Image Pre-trainingcited by 1,147
- 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
- 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
- 2023Vision Transformers Need Registerscited by 844
- 2022Language-driven Semantic Segmentationcited by 711
- 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
- 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
- 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
- 2021Extract Free Dense Labels from CLIPcited by 580
- 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
- 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
- 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
- 2023Side Adapter Network for Open-Vocabulary Semantic Segmentationcited by 296
- 2019Zero-Shot Semantic Segmentationcited by 245
- 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Modelcited by 223
- 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 215
- 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 175
- 2023SCLIP: Rethinking Self-Attention for Dense Vision-Language Inferencecited by 169
- 2020Single-Stage Semantic Segmentation from Image Labelscited by 167
- 2022Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairscited by 130
- 2022ReCo: Retrieve and Co-segment for Zero-shot Transfercited by 115
- 2022Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learningcited by 115
- 2024ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inferencecited by 112
- 2023SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understandingcited by 111
- 2024ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentationcited by 110
- 2023Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervisioncited by 99
- 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 95
- 2023CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentationcited by 94
- 2023Grounding Everything: Emerging Localization Properties in Vision-Language Transformerscited by 89
- 2024DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignmentcited by 79
- 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Groundingcited by 78
- 2017Open Vocabulary Scene Parsingcited by 76
50 most cited of 71 All 71 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Standalone DINOv3 for Training-Free Open-Vocabulary Semantic Segmentation in Remote Sensingcited by 1
- 2026SCI-CLIP: Segment-Centric Inference with Reference Memory for Training-Free Open-Vocabulary Segmentationcited by 0
- 2026SatOV: Restoring Spatial Priors for Training-Free Open-Vocabulary Segmentation in Remote Sensing Imagerycited by 0
- 2026SatOV: Restoring Spatial Priors for Training-Free Open-Vocabulary Segmentation in Remote Sensing Imagery19 Sep 2026
- 2026SCI-CLIP: Segment-Centric Inference with Reference Memory for Training-Free Open-Vocabulary Segmentation6 Aug 2026
- 2026Standalone DINOv3 for Training-Free Open-Vocabulary Semantic Segmentation in Remote Sensing4 Aug 2026