# Look into Person: Joint Body Parsing & Pose Estimation Network and A New Benchmark

arXiv: https://arxiv.org/abs/1804.01984 · HTML: https://mlsanity.com/paper/1804.01984/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2018.

## Built on by (most cited first, 30 of 79)

- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [SegGPT: Segmenting Everything In Context](https://mlsanity.com/paper/2304.03284/) (2023)
- [Taming the Power of Diffusion Models for High-Quality Virtual Try-On with Appearance Flow](https://mlsanity.com/paper/2308.06101/) (2023)
- [PartImageNet: A Large, High-Quality Dataset of Parts](https://mlsanity.com/paper/2112.00933/) (2021)
- [Beyond Appearance: a Semantic Controllable Self-Supervised Learning Framework for Human-Centric Visual Tasks](https://mlsanity.com/paper/2303.17602/) (2023)
- [Deep Hierarchical Semantic Segmentation](https://mlsanity.com/paper/2203.14335/) (2022)
- [UniHCP: A Unified Model for Human-Centric Perceptions](https://mlsanity.com/paper/2303.02936/) (2023)
- [Part-aware Panoptic Segmentation](https://mlsanity.com/paper/2106.06351/) (2021)
- [4D-DRESS: A 4D Dataset of Real-world Human Clothing with Semantic Annotations](https://mlsanity.com/paper/2404.18630/) (2024)
- [Recent Advances in Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspective](https://mlsanity.com/paper/2104.11536/) (2021)
- [PLIP: Language-Image Pre-training for Person Representation Learning](https://mlsanity.com/paper/2305.08386/) (2023)
- [A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering](https://mlsanity.com/paper/2306.06211/) (2023)
- [Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN](https://mlsanity.com/paper/2111.10544/) (2021)
- [HumanBench: Towards General Human-centric Perception with Projector Assisted Pretraining](https://mlsanity.com/paper/2303.05675/) (2023)
- [M3D-VTON: A Monocular-to-3D Virtual Try-On Network](https://mlsanity.com/paper/2108.05126/) (2021)
- [Feature Completion for Occluded Person Re-Identification](https://mlsanity.com/paper/2106.12733/) (2021)
- [Universal Segmentation at Arbitrary Granularity with Language Instruction](https://mlsanity.com/paper/2312.01623/) (2023)
- [CDGNet: Class Distribution Guided Network for Human Parsing](https://mlsanity.com/paper/2111.14173/) (2021)
- [Causal Reasoning Meets Visual Representation Learning: A Prospective Study](https://mlsanity.com/paper/2204.12037/) (2022)
- [Cross-Covariate Gait Recognition: A Benchmark](https://mlsanity.com/paper/2312.14404/) (2023)
- [2D Human Pose Estimation: A Survey](https://mlsanity.com/paper/2204.07370/) (2022)
- [Dressing in the Wild by Watching Dance Videos](https://mlsanity.com/paper/2203.15320/) (2022)
- [Erasing, Transforming, and Noising Defense Network for Occluded Person Re-Identification](https://mlsanity.com/paper/2307.07187/) (2023)
- [Deep Learning Technique for Human Parsing: A Survey and Outlook](https://mlsanity.com/paper/2301.00394/) (2023)
- [CT-Net: Complementary Transfering Network for Garment Transfer with Arbitrary Geometric Changes](https://mlsanity.com/paper/2105.05497/) (2021)
- [Graph-based Person Signature for Person Re-Identifications](https://mlsanity.com/paper/2104.06770/) (2021)
- [Quality-Aware Network for Human Parsing](https://mlsanity.com/paper/2103.05997/) (2021)
- [Robust 3D Garment Digitization from Monocular 2D Images for 3D Virtual Try-On Systems](https://mlsanity.com/paper/2111.15140/) (2021)
- [Occluded Person Re-Identification with Deep Learning: A Survey and Perspectives](https://mlsanity.com/paper/2311.00603/) (2023)

## Abstract (arXiv)

Human parsing and pose estimation have recently received considerable interest due to their substantial application potentials. However, the existing datasets have limited numbers of images and annotations and lack a variety of human appearances and coverage of challenging cases in unconstrained environments. In this paper, we introduce a new benchmark named "Look into Person (LIP)" that provides a significant advancement in terms of scalability, diversity, and difficulty, which are crucial for future developments in human-centric analysis. This comprehensive dataset contains over 50,000 elaborately annotated images with 19 semantic part labels and 16 body joints, which are captured from a broad range of viewpoints, occlusions, and background complexities. Using these rich annotations, we perform detailed analyses of the leading human parsing and pose estimation approaches, thereby obtaining insights into the successes and failures of these methods. To further explore and take advantage of the semantic correlation of these two tasks, we propose a novel joint human parsing and pose estimation network to explore efficient context modeling, which can simultaneously predict parsing and pose with extremely high quality. Furthermore, we simplify the network to solve human parsing by exploring a novel self-supervised structure-sensitive learning approach, which imposes human pose structures into the parsing results without resorting to extra supervision. The dataset, code and models are available at http://www.sysu-hcp.net/lip/.