mlsanityData to 14 Oct 2026

LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentation

arXiv 2503.19777 ↗First posted 25 Mar 2025cs.CV cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

We propose a training-free method for open-vocabulary semantic segmentation using Vision-and-Language Models (VLMs). Our approach enhances the initial per-patch predictions of VLMs through label propagation, which jointly optimizes predictions by incorporating patch-to-patch relationships. Since VLMs are primarily optimized for cross-modal alignment and not for intra-modal similarity, we use a Vision Model (VM) that is observed to better capture these relationships. We address resolution limitations inherent to patch-based encoders by applying label propagation at the pixel level as a refinement step, significantly improving segmentation accuracy near class boundaries. Our method, called LPOSS+, performs inference over the entire image, avoiding window-based processing and thereby capturing contextual interactions across the full image. LPOSS+ achieves state-of-the-art performance among training-free methods, across a diverse set of datasets. Code: https://github.com/vladan-stojnic/LPOSS

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 46 of its 56 references that are in the map. First posted on arXiv 25 Mar 2025.

Builds on

46 of 56 references.

References · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  7. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  8. 2024Depth Anything V2cited by 1,422
  9. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  10. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  11. 2023Vision Transformers Need Registerscited by 871
  12. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
34 more
  1. 2021Extract Free Dense Labels from CLIPcited by 583
  2. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
  3. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 531
  4. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 473
  5. 2019Label Propagation for Deep Semi-supervised Learningcited by 289
  6. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  7. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 219
  8. 2015Conditional Random Fields as Recurrent Neural Networkscited by 214
  9. 2021Boundary IoU: Improving Object-Centric Image Segmentation Evaluationcited by 188
  10. 2023SCLIP: Rethinking Self-Attention for Dense Vision-Language Inferencecited by 170
  11. 2020Single-Stage Semantic Segmentation from Image Labelscited by 167
  12. 2023Towards Open Vocabulary Learning: A Surveycited by 166
  13. 2022SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentationcited by 164
  14. 2022Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairscited by 130
  15. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Modelscited by 119
  16. 2023Aligning Bag of Regions for Open-Vocabulary Object Detectioncited by 119
  17. 2022Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learningcited by 115
  18. 2024ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inferencecited by 113
  19. 2024ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentationcited by 111
  20. 2023Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervisioncited by 99
  21. 2023CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentationcited by 96
  22. 2023Grounding Everything: Emerging Localization Properties in Vision-Language Transformerscited by 91
  23. 2022Perceptual Grouping in Contrastive Vision-Language Modelscited by 82
  24. 2022Unsupervised Object Localization: Observing the Background to Discover Objectscited by 66
  25. 2016Efficient Diffusion on Region Manifolds: Recovering Small Objects with Compact CNN Representationscited by 50
  26. 2023CorrMatch: Label Propagation via Correlation Matching for Semi-Supervised Semantic Segmentationcited by 44
  27. 2023CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Freecited by 39
  28. 2024In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentationcited by 38
  29. 2023ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptationcited by 35
  30. 2022Exploring Visual Interpretability for Contrastive Language-Image Pre-trainingcited by 30
  31. 2024Label Propagation for Zero-shot Classification with Vision-Language Modelscited by 28
  32. 2016Fast, Exact and Multi-Scale Inference for Semantic Image Segmentation with Deep Gaussian CRFscited by 27
  33. 2023What a MESS: Multi-Domain Evaluation of Zero-Shot Semantic Segmentationcited by 24
  34. 2024Co-Segmentation without any Pixel-level Supervision with Application to Large-Scale Sketch Classificationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 16.

21 papers.

Sort:

By area:

  1. 2025TextRegion: Text-Aligned Region Tokens from Frozen Image-Text Modelscited by 7
  2. 2025REGLUE Your Latents with Global and Local Semantics for Entangled Diffusioncited by 5
  3. 2026Taming SAM3 in the Wild: A Concept Bank for Open-Vocabulary Segmentationcited by 4
  4. 2026PEARL: Geometry Aligns Semantics for Training-Free Open-Vocabulary Semantic Segmentationcited by 3
  5. 2026VIP: Visual-guided Prompt Evolution for Efficient Dense Vision-Language Inferencecited by 2
  6. 2026Standalone DINOv3 for Training-Free Open-Vocabulary Semantic Segmentation in Remote Sensingcited by 1
  7. 2025Semi-Supervised Semantic Segmentation via Derivative Label Propagationcited by 0
  8. 2025Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoderscited by 0
  9. 2025ARM: A Learnable, Plug-and-Play Module for CLIP-based Open-vocabulary Semantic Segmentationcited by 0
  10. 2026Retrieve and Segment: Are a Few Examples Enough to Bridge the Supervision Gap in Open-Vocabulary Segmentation?cited by 0
  11. 2026IGLOSS: Image Generation for Lidar Open-vocabulary Semantic Segmentationcited by 0
  12. 2026SPAR: Single-Pass Any-Resolution ViT for Open-vocabulary Segmentationcited by 0
9 more
  1. 2026Direct Segmentation without Logits Optimization for Training-Free Open-Vocabulary Semantic Segmentationcited by 0
  2. 2026MapSR: Prompt-Driven Land Cover Map Super-Resolution via Vision Foundation Modelscited by 0
  3. 2026SENSE: Stereo OpEN Vocabulary SEmantic Segmentationcited by 0
  4. 2026Benchmarking Composed Image Retrieval for Applied Earth Observationcited by 0
  5. 2026Geometry-Preserving Unsupervised Alignment for Heterogeneous Foundation Modelscited by 0
  6. 2026Training-Free Generalized Few-Shot Segmentation through Open-Vocabulary Semantic Arbitrationcited by 0
  7. 2026LASA: A Weak Supervision Method for Open-Vocabulary Scene Sketch Semantic Segmentationcited by 0
  8. 2026ActiveSAM: Fast and Accurate Open-Vocabulary Semantic Segmentation with Frozen SAM 3cited by 0
  9. 2026TGRIP: A Text-Guided Approach to Vehicle Instance Prediction in Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond-Labels: Advancing Open-Vocabulary Segmentation With Vision-Language Models
  2. 2023Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models
  3. 2026dinov3.seg: Open-Vocabulary Semantic Segmentation with DINOv3
  4. 2022Open-vocabulary Semantic Segmentation with Frozen Vision-Language Models
  5. 2024OpenDAS: Open-Vocabulary Domain Adaptation for 2D and 3D Segmentation
  6. 2025From Open-Vocabulary to Vocabulary-Free Semantic Segmentation
  7. 2024LMSeg: Unleashing the Power of Large-Scale Models for Open-Vocabulary Semantic Segmentation
  8. 2026OV-Stitcher: A Global Context-Aware Framework for Training-Free Open-Vocabulary Semantic Segmentation