# Joint Multi-Person Pose Estimation and Semantic Part Segmentation

arXiv: https://arxiv.org/abs/1708.03383 · HTML: https://mlsanity.com/paper/1708.03383/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2017.

## Built on by (most cited first, 30 of 41)

- [Sapiens: Foundation for Human Vision Models](https://mlsanity.com/paper/2408.12569/) (2024)
- [PartImageNet: A Large, High-Quality Dataset of Parts](https://mlsanity.com/paper/2112.00933/) (2021)
- [Deep Hierarchical Semantic Segmentation](https://mlsanity.com/paper/2203.14335/) (2022)
- [OPD: Single-view 3D Openable Part Detection](https://mlsanity.com/paper/2203.16421/) (2022)
- [Recent Advances in Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspective](https://mlsanity.com/paper/2104.11536/) (2021)
- [PLIP: Language-Image Pre-training for Person Representation Learning](https://mlsanity.com/paper/2305.08386/) (2023)
- [Learning Multiple Dense Prediction Tasks from Partially Annotated Data](https://mlsanity.com/paper/2111.14893/) (2021)
- [Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computing](https://mlsanity.com/paper/2210.15889/) (2022)
- [Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing](https://mlsanity.com/paper/2103.04570/) (2021)
- [Cross-Covariate Gait Recognition: A Benchmark](https://mlsanity.com/paper/2312.14404/) (2023)
- [2D Human Pose Estimation: A Survey](https://mlsanity.com/paper/2204.07370/) (2022)
- [Deep Learning Technique for Human Parsing: A Survey and Outlook](https://mlsanity.com/paper/2301.00394/) (2023)
- [Learning Part Segmentation through Unsupervised Domain Adaptation from Synthetic Vehicles](https://mlsanity.com/paper/2103.14098/) (2021)
- [Quality-Aware Network for Human Parsing](https://mlsanity.com/paper/2103.05997/) (2021)
- [Exploring More from Multiple Gait Modalities for Human Identification](https://mlsanity.com/paper/2412.11495/) (2024)
- [Part-Based Models Improve Adversarial Robustness](https://mlsanity.com/paper/2209.09117/) (2022)
- [FLOAT: Factorized Learning of Object Attributes for Improved Multi-object Multi-part Scene Parsing](https://mlsanity.com/paper/2203.16168/) (2022)
- [From Frames to Sequences: Temporally Consistent Human-Centric Dense Prediction](https://mlsanity.com/paper/2602.01661/) (2026)
- [3D Pose Estimation and Future Motion Prediction from 2D Images](https://mlsanity.com/paper/2111.13285/) (2021)
- [Deep Virtual Markers for Articulated 3D Shapes](https://mlsanity.com/paper/2108.09000/) (2021)
- [AIMS: All-Inclusive Multi-Level Segmentation](https://mlsanity.com/paper/2305.17768/) (2023)
- [Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer](https://mlsanity.com/paper/2101.10620/) (2021)
- [Active Coarse-to-Fine Segmentation of Moveable Parts from Real Images](https://mlsanity.com/paper/2303.11530/) (2023)
- [Human Body Pose Estimation for Gait Identification: A Comprehensive Survey of Datasets and Models](https://mlsanity.com/paper/2305.13765/) (2023)
- [End-to-end One-shot Human Parsing](https://mlsanity.com/paper/2105.01241/) (2021)
- [Single Person Pose Estimation: A Survey](https://mlsanity.com/paper/2109.10056/) (2021)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)
- [Human Modelling and Pose Estimation Overview](https://mlsanity.com/paper/2406.19290/) (2024)
- [Quality-Aware Network for Face Parsing](https://mlsanity.com/paper/2106.07368/) (2021)
- [Describe me if you can! Characterized Instance-level Human Parsing](https://mlsanity.com/paper/2201.09594/) (2022)

## Abstract (arXiv)

Human pose estimation and semantic part segmentation are two complementary tasks in computer vision. In this paper, we propose to solve the two tasks jointly for natural multi-person images, in which the estimated pose provides object-level shape prior to regularize part segments while the part-level segments constrain the variation of pose locations. Specifically, we first train two fully convolutional neural networks (FCNs), namely Pose FCN and Part FCN, to provide initial estimation of pose joint potential and semantic part potential. Then, to refine pose joint location, the two types of potentials are fused with a fully-connected conditional random field (FCRF), where a novel segment-joint smoothness term is used to encourage semantic and spatial consistency between parts and joints. To refine part segments, the refined pose and the original part potential are integrated through a Part FCN, where the skeleton feature from pose serves as additional regularization cues for part segments. Finally, to reduce the complexity of the FCRF, we induce human detection boxes and infer the graph inside each box, making the inference forty times faster. Since there's no dataset that contains both part segments and pose labels, we extend the PASCAL VOC part dataset with human pose joints and perform extensive experiments to compare our method against several most recent strategies. We show that on this dataset our algorithm surpasses competing methods by a large margin in both tasks.