mlsanityData to 7 Oct 2026

Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentation

arXiv 2501.09688 ↗First posted 16 Jan 2025cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Open-Vocabulary Part Segmentation (OVPS) is an emerging field for recognizing fine-grained parts in unseen categories. We identify two primary challenges in OVPS: (1) the difficulty in aligning part-level image-text correspondence, and (2) the lack of structural understanding in segmenting object parts. To address these issues, we propose PartCATSeg, a novel framework that integrates object-aware part-level cost aggregation, compositional loss, and structural guidance from DINO. Our approach employs a disentangled cost aggregation strategy that handles object and part-level costs separately, enhancing the precision of part-level segmentation. We also introduce a compositional loss to better capture part-object relationships, compensating for the limited part annotations. Additionally, structural guidance from DINO features improves boundary delineation and inter-part understanding. Extensive experiments on Pascal-Part-116, ADE20K-Part-234, and PartImageNet datasets demonstrate that our method significantly outperforms state-of-the-art approaches, setting a new baseline for robust generalization to unseen part categories.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 64 of its 84 references that are in the map. First posted on arXiv 16 Jan 2025.

Builds on

64 of 84 references.

References · 64

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2017Mask R-CNNcited by 7,259
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  9. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  12. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
38 more
  1. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  2. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  3. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  4. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  5. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  6. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  7. 2022Language-driven Semantic Segmentationcited by 711
  8. 2021Extract Free Dense Labels from CLIPcited by 580
  9. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
  10. 2021Image Segmentation Using Text and Image Promptscited by 509
  11. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
  12. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  13. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
  14. 2019Zero-Shot Semantic Segmentationcited by 245
  15. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 230
  16. 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Modelcited by 223
  17. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 215
  18. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 199
  19. 2022ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentationcited by 178
  20. 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
  21. 2018Neighbourhood Consensus Networkscited by 173
  22. 2014Evaluation of Output Embeddings for Fine-Grained Image Classificationcited by 166
  23. 2015Label-Embedding for Image Classificationcited by 164
  24. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  25. 2019GLU-Net: Global-Local Universal Network for Dense Flow and Correspondencescited by 139
  26. 2022Open-Vocabulary Universal Image Segmentation with MaskCLIPcited by 131
  27. 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
  28. 2021PartImageNet: A Large, High-Quality Dataset of Partscited by 107
  29. 2023HomeRobot: Open-Vocabulary Mobile Manipulationcited by 104
  30. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  31. 2020A3: Accelerating Attention Mechanisms in Neural Networks with Approximationcited by 97
  32. 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 95
  33. 2021CATs: Cost Aggregation Transformers for Visual Correspondencecited by 77
  34. 2017Open Vocabulary Scene Parsingcited by 76
  35. 2021Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labelscited by 74
  36. 2023SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulationcited by 66
  37. 2021Unsupervised Part Discovery from Contrastive Reconstructioncited by 57
  38. 2019Hyperpixel Flow: Semantic Correspondence with Multi-layer Neural Featurescited by 56

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2026PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentationcited by 4
  2. 2025PartCo: Part-Level Correspondence Priors Enhance Category Discoverycited by 2
  3. 2025LangHOPS: Language Grounded Hierarchical Open-Vocabulary Part Segmentationcited by 1
  4. 2025PowerCLIP: Powerset Alignment for Contrastive Pre-Trainingcited by 1