mlsanityData to 7 Oct 2026

PEARL: Geometry Aligns Semantics for Training-Free Open-Vocabulary Semantic Segmentation

arXiv 2603.21528 ↗First posted 23 Mar 2026cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Training-free open-vocabulary semantic segmentation (OVSS) promises rapid adaptation to new label sets without retraining. Yet, many methods rely on heavy post-processing or handle text and vision in isolation, leaving cross-modal geometry underutilized. Others introduce auxiliary vision backbones or multi-model pipelines, which increase complexity and latency while compromising design simplicity. We present PEARL, textbfunderlineProcrusttextbfunderlinees textbfunderlinealignment with text-awatextbfunderlinere textbfunderlineLaplacian propagation, a compact two-step inference that follows an align-then-propagate principle. The Procrustes alignment step performs an orthogonal projection inside the last self-attention block, rotating keys toward the query subspace via a stable polar iteration. The text-aware Laplacian propagation then refines per-pixel logits on a small grid through a confidence-weighted, text-guided graph solve: text provides both a data-trust signal and neighbor gating, while image gradients preserve boundaries. In this work, our method is fully training-free, plug-and-play, and uses only fixed constants, adding minimal latency with a small per-head projection and a few conjugate-gradient steps. Our approach, PEARL, sets a new state-of-the-art in training-free OVSS without extra data or auxiliary backbones across standard benchmarks, achieving superior performance under both with-background and without-background protocols.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 71 of its 82 references that are in the map. First posted on arXiv 23 Mar 2026.

Builds on

71 of 82 references.

References · 71

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2023Segment Anythingcited by 8,158
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  9. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  12. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
38 more
  1. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  2. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  3. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  4. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  5. 2025DINOv3cited by 1,297
  6. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  7. 2021Grounded Language-Image Pre-trainingcited by 1,147
  8. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  9. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  10. 2023Vision Transformers Need Registerscited by 844
  11. 2022Language-driven Semantic Segmentationcited by 711
  12. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  13. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  14. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  15. 2021Extract Free Dense Labels from CLIPcited by 580
  16. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
  17. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  18. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
  19. 2023Side Adapter Network for Open-Vocabulary Semantic Segmentationcited by 296
  20. 2019Zero-Shot Semantic Segmentationcited by 245
  21. 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Modelcited by 223
  22. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 215
  23. 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 175
  24. 2023SCLIP: Rethinking Self-Attention for Dense Vision-Language Inferencecited by 169
  25. 2020Single-Stage Semantic Segmentation from Image Labelscited by 167
  26. 2022Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairscited by 130
  27. 2022ReCo: Retrieve and Co-segment for Zero-shot Transfercited by 115
  28. 2022Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learningcited by 115
  29. 2024ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inferencecited by 112
  30. 2023SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understandingcited by 111
  31. 2024ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentationcited by 110
  32. 2023Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervisioncited by 99
  33. 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 95
  34. 2023CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentationcited by 94
  35. 2023Grounding Everything: Emerging Localization Properties in Vision-Language Transformerscited by 89
  36. 2024DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignmentcited by 79
  37. 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Groundingcited by 78
  38. 2017Open Vocabulary Scene Parsingcited by 76

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.