mlsanityData to 7 Oct 2026

Personalized OVSS: Understanding Personal Concept in Open-Vocabulary Semantic Segmentation

arXiv 2507.11030 ↗First posted 15 Jul 2025cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

While open-vocabulary semantic segmentation (OVSS) can segment an image into semantic regions based on arbitrarily given text descriptions even for classes unseen during training, it fails to understand personal texts (e.g., `my mug cup') for segmenting regions of specific interest to users. This paper addresses challenges like recognizing `my mug cup' among `multiple mug cups'. To overcome this challenge, we introduce a novel task termed textitpersonalized open-vocabulary semantic segmentation and propose a text prompt tuning-based plug-in method designed to recognize personal visual concepts using a few pairs of images and masks, while maintaining the performance of the original OVSS. Based on the observation that reducing false predictions is essential when applying text prompt tuning to this task, our proposed method employs `negative mask proposal' that captures visual concepts other than the personalized concept. We further improve the performance by enriching the representation of text prompts by injecting visual embeddings of the personal concept into them. This approach enhances personalized OVSS without compromising the original OVSS performance. We demonstrate the superiority of our method on our newly established benchmarks for this task, including FSSper, CUBper, and ADEper.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 39 of its 46 references that are in the map. First posted on arXiv 15 Jul 2025.

Builds on

39 of 46 references.

References · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023Segment Anythingcited by 8,158
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  6. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  7. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  8. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  9. 2022Language-driven Semantic Segmentationcited by 711
  10. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  11. 2023Segment Everything Everywhere All at Oncecited by 546
  12. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 446
27 more
  1. 2023Side Adapter Network for Open-Vocabulary Semantic Segmentationcited by 296
  2. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  3. 2023Personalize Segment Anything Model with One Shotcited by 250
  4. 2019Zero-Shot Semantic Segmentationcited by 245
  5. 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Modelcited by 223
  6. 2023Class-Incremental Learning: A Surveycited by 209
  7. 2023SegGPT: Segmenting Everything In Contextcited by 193
  8. 2022SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentationcited by 162
  9. 2019FSS-1000: A 1000-Class Dataset for Few-Shot Segmentationcited by 160
  10. 2023Matcher: Segment Anything with One Shot Using All-Purpose Feature Matchingcited by 142
  11. 2021Few-Shot Segmentation via Cycle-Consistent Transformercited by 133
  12. 2022Open-Vocabulary Universal Image Segmentation with MaskCLIPcited by 131
  13. 2021Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformercited by 116
  14. 2022"This is my unicorn, Fluffy": Personalizing frozen vision-language representationscited by 95
  15. 2020Context-aware Feature Generation for Zero-shot Semantic Segmentationcited by 94
  16. 2023FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentationcited by 94
  17. 2021Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labelingcited by 84
  18. 2023TCP:Textual-based Class-aware Prompt tuning for Visual-Language Modelcited by 75
  19. 2021Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentationcited by 53
  20. 2023Towards In-context Scene Understandingcited by 45
  21. 2022VoP: Text-Video Co-operative Prompt Tuning for Cross-Modal Retrievalcited by 36
  22. 2023Distribution-Aware Prompt Tuning for Vision-Language Modelscited by 35
  23. 2023SLiMe: Segment Like Mecited by 34
  24. 2022A Strong Baseline for Generalized Few-Shot Semantic Segmentationcited by 22
  25. 2024kNN-CLIP: Retrieval Enables Training-Free Segmentation on Continually Expanding Large Vocabulariescited by 20
  26. 2023SEGIC: Unleashing the Emergent Correspondence for In-Context Segmentationcited by 19
  27. 2022tSF: Transformer-based Semantic Filter for Few-Shot Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreementcited by 0