# Unsupervised Part Segmentation through Disentangling Appearance and Shape

arXiv: https://arxiv.org/abs/2105.12405 · HTML: https://mlsanity.com/paper/2105.12405/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 26 May 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [Self-supervised Single-view 3D Reconstruction via Semantic Consistency](https://mlsanity.com/paper/2003.06473/) (2020)
- [Unsupervised Learning of Object Landmarks through Conditional Image Generation](https://mlsanity.com/paper/1806.07823/) (2018)
- [HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition](https://mlsanity.com/paper/1603.01249/) (2016)
- [SCOPS: Self-Supervised Co-Part Segmentation](https://mlsanity.com/paper/1905.01298/) (2019)
- [Unsupervised Discovery of Object Landmarks as Structural Representations](https://mlsanity.com/paper/1804.04412/) (2018)
- [Unsupervised Part-Based Disentangling of Object Shape and Appearance](https://mlsanity.com/paper/1903.06946/) (2019)
- [Unsupervised learning of object landmarks by factorized spatial embeddings](https://mlsanity.com/paper/1705.02193/) (2017)
- [Deep Feature Factorization For Concept Discovery](https://mlsanity.com/paper/1806.10206/) (2018)
- [Learning Deep Representation for Face Alignment with Auxiliary Attributes](https://mlsanity.com/paper/1408.3967/) (2014)
- [Deep Deformation Network for Object Landmark Localization](https://mlsanity.com/paper/1605.01014/) (2016)
- [One-shot Face Recognition by Promoting Underrepresented Classes](https://mlsanity.com/paper/1707.05574/) (2017)
- [Unsupervised Landmark Learning from Unpaired Data](https://mlsanity.com/paper/2007.01053/) (2020)
- [Deep Structured Prediction for Facial Landmark Detection](https://mlsanity.com/paper/2010.09035/) (2020)

## Built on by (most cited first, 29 of 29)

- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation](https://mlsanity.com/paper/2112.01036/) (2021)
- [PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation](https://mlsanity.com/paper/2301.00954/) (2023)
- [Learning to Refactor Action and Co-occurrence Features for Temporal Action Localization](https://mlsanity.com/paper/2206.11493/) (2022)
- [Recognizing Object by Components with Human Prior Knowledge Enhances Adversarial Robustness of Deep Neural Networks](https://mlsanity.com/paper/2212.01806/) (2022)
- [AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypoints](https://mlsanity.com/paper/2205.10636/) (2022)
- [U3DS3: Unsupervised 3D Semantic Scene Segmentation](https://mlsanity.com/paper/2311.06018/) (2023)
- [Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention](https://mlsanity.com/paper/2304.03282/) (2023)
- [Compositor: Bottom-up Clustering and Compositing for Robust Part and Object Segmentation](https://mlsanity.com/paper/2306.07404/) (2023)
- [HP-Capsule: Unsupervised Face Part Discovery by Hierarchical Parsing Capsule Network](https://mlsanity.com/paper/2203.10699/) (2022)
- [PartImageNet++ Dataset: Scaling up Part-based Models for Robust Recognition](https://mlsanity.com/paper/2407.10918/) (2024)
- [Retriever: Learning Content-Style Representation as a Token-Level Bipartite Graph](https://mlsanity.com/paper/2202.12307/) (2022)
- [TUSK: Task-Agnostic Unsupervised Keypoints](https://mlsanity.com/paper/2206.08460/) (2022)
- [Extracting Semantic Knowledge from GANs with Unsupervised Learning](https://mlsanity.com/paper/2211.16710/) (2022)
- [G4Seg: Generation for Inexact Segmentation Refinement with Diffusion Models](https://mlsanity.com/paper/2506.01539/) (2025)
- [Learning Part Segmentation from Synthetic Animals](https://mlsanity.com/paper/2311.18661/) (2023)
- [WPS-SAM: Towards Weakly-Supervised Part Segmentation with Foundation Models](https://mlsanity.com/paper/2407.10131/) (2024)
- [CALICO: Part-Focused Semantic Co-Segmentation with Large Vision-Language Models](https://mlsanity.com/paper/2412.19331/) (2024)
- [Graphics Capsule: Learning Hierarchical 3D Face Representations from 2D Images](https://mlsanity.com/paper/2303.10896/) (2023)
- [Parsing Objects at a Finer Granularity: A Survey](https://mlsanity.com/paper/2212.13693/) (2022)
- [Unsupervised Part Discovery via Dual Representation Alignment](https://mlsanity.com/paper/2408.08108/) (2024)
- [Deep Learning for Human Parsing: A Survey](https://mlsanity.com/paper/2301.12416/) (2023)
- [Bottom-Up Scattering Information Perception Network for SAR target recognition](https://mlsanity.com/paper/2504.04780/) (2025)
- [Unsupervised Part Discovery via Descriptor-Based Masked Image Restoration with Optimized Constraints](https://mlsanity.com/paper/2507.11985/) (2025)
- [LatentKeypointGAN: Controlling Images via Latent Keypoints -- Extended Abstract](https://mlsanity.com/paper/2205.03448/) (2022)
- [Towards Imbalanced Motion: Part-Decoupling Network for Video Portrait Segmentation](https://mlsanity.com/paper/2307.16565/) (2023)
- [DualContrast: Unsupervised Disentangling of Content and Transformations with Implicit Parameterization](https://mlsanity.com/paper/2405.16796/) (2024)
- [PartImageNet++ Dataset: Enhancing Visual Models with High-Quality Part Annotations](https://mlsanity.com/paper/2601.01454/) (2026)

## Abstract (arXiv)

We study the problem of unsupervised discovery and segmentation of object parts, which, as an intermediate local representation, are capable of finding intrinsic object structure and providing more explainable recognition results. Recent unsupervised methods have greatly relaxed the dependency on annotated data which are costly to obtain, but still rely on additional information such as object segmentation mask or saliency map. To remove such a dependency and further improve the part segmentation performance, we develop a novel approach by disentangling the appearance and shape representations of object parts followed with reconstruction losses without using additional object mask information. To avoid degenerated solutions, a bottleneck block is designed to squeeze and expand the appearance representation, leading to a more effective disentanglement between geometry and appearance. Combined with a self-supervised part classification loss and an improved geometry concentration constraint, we can segment more consistent parts with semantic meanings. Comprehensive experiments on a wide variety of objects such as face, bird, and PASCAL VOC objects demonstrate the effectiveness of the proposed method.