mlsanityData to 14 Oct 2026

Understanding Multi-Granularity for Open-Vocabulary Part Segmentation

arXiv 2406.11384 ↗First posted 17 Jun 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Open-vocabulary part segmentation (OVPS) is an emerging research area focused on segmenting fine-grained entities using diverse and previously unseen vocabularies. Our study highlights the inherent complexities of part segmentation due to intricate boundaries and diverse granularity, reflecting the knowledge-based nature of part identification. To address these challenges, we propose PartCLIPSeg, a novel framework utilizing generalized parts and object-level contexts to mitigate the lack of generalization in fine-grained parts. PartCLIPSeg integrates competitive part relationships and attention control, alleviating ambiguous boundaries and underrepresented parts. Experimental results demonstrate that PartCLIPSeg outperforms existing state-of-the-art OVPS methods, offering refined segmentation and an advanced understanding of part relationships within images. Through extensive experiments, our model demonstrated a significant improvement over the state-of-the-art models on the Pascal-Part-116, ADE20K-Part-234, and PartImageNet datasets.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 50 of its 60 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

50 of 60 references.

References · 50

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Segment Anythingcited by 8,234
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  5. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  6. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  7. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  8. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  9. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
  10. 2022Language-driven Semantic Segmentationcited by 717
  11. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  12. 2021Extract Free Dense Labels from CLIPcited by 583
38 more
  1. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
  2. 2021Image Segmentation Using Text and Image Promptscited by 512
  3. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 473
  4. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  5. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  6. 2023Side Adapter Network for Open-Vocabulary Semantic Segmentationcited by 300
  7. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 282
  8. 2019Zero-Shot Semantic Segmentationcited by 245
  9. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 232
  10. 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Modelcited by 223
  11. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 219
  12. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 201
  13. 2021Decoupling Zero-Shot Semantic Segmentationcited by 199
  14. 2021Boundary IoU: Improving Object-Centric Image Segmentation Evaluationcited by 188
  15. 2023Semantic-SAM: Segment and Recognize Anything at Any Granularitycited by 185
  16. 2022ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentationcited by 178
  17. 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
  18. 2023Grounded Text-to-Image Synthesis with Attention Refocusingcited by 164
  19. 2023Dense Text-to-Image Generation with Attention Modulationcited by 163
  20. 2023DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Modelscited by 162
  21. 2022Open-Vocabulary Universal Image Segmentation with MaskCLIPcited by 132
  22. 2023Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusioncited by 131
  23. 2021PartImageNet: A Large, High-Quality Dataset of Partscited by 107
  24. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  25. 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 97
  26. 2023A-STAR: Test-time Attention Segregation and Retention for Text-to-image Synthesiscited by 78
  27. 2017Open Vocabulary Scene Parsingcited by 76
  28. 2021Unsupervised Part Discovery from Contrastive Reconstructioncited by 57
  29. 2021Part-aware Panoptic Segmentationcited by 56
  30. 2023Going Denser with Open-Vocabulary Part Segmentationcited by 51
  31. 2023Open-Vocabulary Segmentation with Semantic-Assisted Calibrationcited by 49
  32. 2023Global Knowledge Calibration for Fast Open-Vocabulary Segmentationcited by 40
  33. 2023OV-PARTS: Towards Open-Vocabulary Part Segmentationcited by 31
  34. 2022FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulationcited by 24
  35. 2023PDiscoNet: Semantically consistent part discovery for fine-grained recognitioncited by 21
  36. 2023Separate-and-Enhance: Compositional Finetuning for Text2Image Diffusion Modelscited by 19
  37. 2023Towards Open-World Segmentation of Partscited by 15
  38. 2023Compositor: Bottom-up Clustering and Compositing for Robust Part and Object Segmentationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentationcited by 5
  2. 2025Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentationcited by 4
  3. 2026SceneParser: Hierarchical Scene Parsing for Visual Semantics Understandingcited by 0