# Joint Object and Part Segmentation using Deep Learned Potentials

arXiv: https://arxiv.org/abs/1505.00276 · HTML: https://mlsanity.com/paper/1505.00276/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2015.

## Built on by (most cited first, 23 of 23)

- [PartImageNet: A Large, High-Quality Dataset of Parts](https://mlsanity.com/paper/2112.00933/) (2021)
- [Repurposing GANs for One-shot Semantic Part Segmentation](https://mlsanity.com/paper/2103.04379/) (2021)
- [Part-aware Panoptic Segmentation](https://mlsanity.com/paper/2106.06351/) (2021)
- [OPD: Single-view 3D Openable Part Detection](https://mlsanity.com/paper/2203.16421/) (2022)
- [InvPT++: Inverted Pyramid Multi-Task Transformer for Visual Scene Understanding](https://mlsanity.com/paper/2306.04842/) (2023)
- [Learning Part Segmentation through Unsupervised Domain Adaptation from Synthetic Vehicles](https://mlsanity.com/paper/2103.14098/) (2021)
- [PartGLEE: A Foundation Model for Recognizing and Parsing Any Objects](https://mlsanity.com/paper/2407.16696/) (2024)
- [3D-MuPPET: 3D Multi-Pigeon Pose Estimation and Tracking](https://mlsanity.com/paper/2308.15316/) (2023)
- [Compositor: Bottom-up Clustering and Compositing for Robust Part and Object Segmentation](https://mlsanity.com/paper/2306.07404/) (2023)
- [FLOAT: Factorized Learning of Object Attributes for Improved Multi-object Multi-part Scene Parsing](https://mlsanity.com/paper/2203.16168/) (2022)
- [Active Coarse-to-Fine Segmentation of Moveable Parts from Real Images](https://mlsanity.com/paper/2303.11530/) (2023)
- [SPIN: Hierarchical Segmentation with Subpart Granularity in Natural Images](https://mlsanity.com/paper/2407.09686/) (2024)
- [Learning Part Segmentation from Synthetic Animals](https://mlsanity.com/paper/2311.18661/) (2023)
- [Learning to Annotate Part Segmentation with Gradient Matching](https://mlsanity.com/paper/2211.03003/) (2022)
- [Parsing Objects at a Finer Granularity: A Survey](https://mlsanity.com/paper/2212.13693/) (2022)
- [Inferring and Leveraging Parts from Object Shape for Improving Semantic Image Synthesis](https://mlsanity.com/paper/2305.19547/) (2023)
- [OP3DSG: Open-Vocabulary Part-Aware 3D Scene Graph Generation for Real-World Environments](https://mlsanity.com/paper/2606.29786/) (2026)
- [Annotating Motion Primitives for Simplifying Action Search in Reinforcement Learning](https://mlsanity.com/paper/2102.12017/) (2021)
- [Graph-Boosted Attentive Network for Semantic Body Parsing](https://mlsanity.com/paper/2407.05924/) (2024)
- [From Pixels to Objects: A Hierarchical Approach for Part and Object Segmentation Using Local and Global Aggregation](https://mlsanity.com/paper/2409.01353/) (2024)
- [OLAF: A Plug-and-Play Framework for Enhanced Multi-object Multi-part Scene Parsing](https://mlsanity.com/paper/2411.02858/) (2024)
- [Dictionary-based Framework for Interpretable and Consistent Object Parsing](https://mlsanity.com/paper/2502.19540/) (2025)
- [One-shot In-context Part Segmentation](https://mlsanity.com/paper/2503.01144/) (2025)

## Abstract (arXiv)

Segmenting semantic objects from images and parsing them into their respective semantic parts are fundamental steps towards detailed object understanding in computer vision. In this paper, we propose a joint solution that tackles semantic object and part segmentation simultaneously, in which higher object-level context is provided to guide part segmentation, and more detailed part-level localization is utilized to refine object segmentation. Specifically, we first introduce the concept of semantic compositional parts (SCP) in which similar semantic parts are grouped and shared among different objects. A two-channel fully convolutional network (FCN) is then trained to provide the SCP and object potentials at each pixel. At the same time, a compact set of segments can also be obtained from the SCP predictions of the network. Given the potentials and the generated segments, in order to explore long-range context, we finally construct an efficient fully connected conditional random field (FCRF) to jointly predict the final object and part labels. Extensive evaluation on three different datasets shows that our approach can mutually enhance the performance of object and part segmentation, and outperforms the current state-of-the-art on both tasks.