# OV-PARTS: Towards Open-Vocabulary Part Segmentation

arXiv: https://arxiv.org/abs/2310.05107 · HTML: https://mlsanity.com/paper/2310.05107/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 38 references that are in the map.
- First posted on arXiv 8 Oct 2023.

## Builds on (most cited first, 30 of 32)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [CLIPort: What and Where Pathways for Robotic Manipulation](https://mlsanity.com/paper/2109.12098/) (2021)
- [Extract Free Dense Labels from CLIP](https://mlsanity.com/paper/2112.01071/) (2021)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.04803/) (2023)
- [Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts](https://mlsanity.com/paper/1406.2031/) (2014)
- [CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2303.11797/) (2023)
- [Decoupling Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/2112.07910/) (2021)
- [Attention to Scale: Scale-aware Semantic Image Segmentation](https://mlsanity.com/paper/1511.03339/) (2015)
- [EditGAN: High-Precision Semantic Image Editing](https://mlsanity.com/paper/2111.03186/) (2021)
- [PhraseCut: Language-based Image Segmentation in the Wild](https://mlsanity.com/paper/2008.01187/) (2020)
- [Learning Object-Language Alignments for Open-Vocabulary Object Detection](https://mlsanity.com/paper/2211.14843/) (2022)
- [PLOT: Prompt Learning with Optimal Transport for Vision-Language Models](https://mlsanity.com/paper/2210.01253/) (2022)
- [Part-aware Panoptic Segmentation](https://mlsanity.com/paper/2106.06351/) (2021)
- [Going Denser with Open-Vocabulary Part Segmentation](https://mlsanity.com/paper/2305.11173/) (2023)
- [Panoptic-PartFormer: Learning a Unified Model for Panoptic Part Segmentation](https://mlsanity.com/paper/2204.04655/) (2022)
- [Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing](https://mlsanity.com/paper/2103.04570/) (2021)
- [GMNet: Graph Matching Network for Large Scale Part Semantic Segmentation in the Wild](https://mlsanity.com/paper/2007.09073/) (2020)
- [Semantic Part Segmentation using Compositional Model combining Shape and Appearance](https://mlsanity.com/paper/1412.6124/) (2014)
- [Towards Open-World Segmentation of Parts](https://mlsanity.com/paper/2305.16804/) (2023)

## Built on by (most cited first, 30 of 31)

- [SAMPart3D: Segment Any Part in 3D Objects](https://mlsanity.com/paper/2411.07184/) (2024)
- [One-Shot Open Affordance Learning with Foundation Models](https://mlsanity.com/paper/2311.17776/) (2023)
- [Learning Precise Affordances from Egocentric Videos for Robotic Manipulation](https://mlsanity.com/paper/2408.10123/) (2024)
- [Search3D: Hierarchical Open-Vocabulary 3D Segmentation](https://mlsanity.com/paper/2409.18431/) (2024)
- [MMR: A Large-scale Benchmark Dataset for Multi-target and Multi-granularity Reasoning Segmentation](https://mlsanity.com/paper/2503.13881/) (2025)
- [Open Panoramic Segmentation](https://mlsanity.com/paper/2407.02685/) (2024)
- [PartGLEE: A Foundation Model for Recognizing and Parsing Any Objects](https://mlsanity.com/paper/2407.16696/) (2024)
- [Reasoning Segmentation for Images and Videos: A Survey](https://mlsanity.com/paper/2505.18816/) (2025)
- [Long-Tailed 3D Detection via Multi-Modal Fusion](https://mlsanity.com/paper/2312.10986/) (2023)
- [Vision-Language Model for Object Detection and Segmentation: A Review and Evaluation](https://mlsanity.com/paper/2504.09480/) (2025)
- [CapeX: Category-Agnostic Pose Estimation from Textual Point Explanation](https://mlsanity.com/paper/2406.00384/) (2024)
- [SMITE: Segment Me In TimE](https://mlsanity.com/paper/2410.18538/) (2024)
- [Task-aligned Part-aware Panoptic Segmentation through Joint Object-Part Representations](https://mlsanity.com/paper/2406.10114/) (2024)
- [SPIN: Hierarchical Segmentation with Subpart Granularity in Natural Images](https://mlsanity.com/paper/2407.09686/) (2024)
- [DreamBeast: Distilling 3D Fantastical Animals with Part-Aware Knowledge Transfer](https://mlsanity.com/paper/2409.08271/) (2024)
- [PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentation](https://mlsanity.com/paper/2603.17520/) (2026)
- [WPS-SAM: Towards Weakly-Supervised Part Segmentation with Foundation Models](https://mlsanity.com/paper/2407.10131/) (2024)
- [CALICO: Part-Focused Semantic Co-Segmentation with Large Vision-Language Models](https://mlsanity.com/paper/2412.19331/) (2024)
- [Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentation](https://mlsanity.com/paper/2501.09688/) (2025)
- [Understanding Multi-Granularity for Open-Vocabulary Part Segmentation](https://mlsanity.com/paper/2406.11384/) (2024)
- [3DFroMLLM: 3D Prototype Generation only from Pretrained Multimodal LLMs](https://mlsanity.com/paper/2508.08821/) (2025)
- [LangHOPS: Language Grounded Hierarchical Open-Vocabulary Part Segmentation](https://mlsanity.com/paper/2510.25263/) (2025)
- [Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spaces](https://mlsanity.com/paper/2605.15753/) (2026)
- [Vision Harnessing Agent for Open Ad-hoc Segmentation](https://mlsanity.com/paper/2605.19410/) (2026)
- [OP3DSG: Open-Vocabulary Part-Aware 3D Scene Graph Generation for Real-World Environments](https://mlsanity.com/paper/2606.29786/) (2026)
- [OLAF: A Plug-and-Play Framework for Enhanced Multi-object Multi-part Scene Parsing](https://mlsanity.com/paper/2411.02858/) (2024)
- [Guided SAM: Label-Efficient Part Segmentation](https://mlsanity.com/paper/2501.07434/) (2025)
- [One-shot In-context Part Segmentation](https://mlsanity.com/paper/2503.01144/) (2025)
- [Talking Points: Describing and Localizing Pixels](https://mlsanity.com/paper/2510.14583/) (2025)
- [Hierarchical Instance Tracking to Balance Privacy Preservation with Accessible Information](https://mlsanity.com/paper/2512.10102/) (2025)

## Abstract (arXiv)

Segmenting and recognizing diverse object parts is a crucial ability in applications spanning various computer vision and robotic tasks. While significant progress has been made in object-level Open-Vocabulary Semantic Segmentation (OVSS), i.e., segmenting objects with arbitrary text, the corresponding part-level research poses additional challenges. Firstly, part segmentation inherently involves intricate boundaries, while limited annotated data compounds the challenge. Secondly, part segmentation introduces an open granularity challenge due to the diverse and often ambiguous definitions of parts in the open world. Furthermore, the large-scale vision and language models, which play a key role in the open vocabulary setting, struggle to recognize parts as effectively as objects. To comprehensively investigate and tackle these challenges, we propose an Open-Vocabulary Part Segmentation (OV-PARTS) benchmark. OV-PARTS includes refined versions of two publicly available datasets: Pascal-Part-116 and ADE20K-Part-234. And it covers three specific tasks: Generalized Zero-Shot Part Segmentation, Cross-Dataset Part Segmentation, and Few-Shot Part Segmentation, providing insights into analogical reasoning, open granularity and few-shot adapting abilities of models. Moreover, we analyze and adapt two prevailing paradigms of existing object-level OVSS methods for OV-PARTS. Extensive experimental analysis is conducted to inspire future research in leveraging foundational models for OV-PARTS. The code and dataset are available at https://github.com/OpenRobotLab/OV_PARTS.