# Unsupervised Discovery of Parts, Structure, and Dynamics

arXiv: https://arxiv.org/abs/1903.05136 · HTML: https://mlsanity.com/paper/1903.05136/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2019.

## Built on by (most cited first, 20 of 20)

- [PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2103.09504/) (2021)
- [Repurposing GANs for One-shot Semantic Part Segmentation](https://mlsanity.com/paper/2103.04379/) (2021)
- [Unsupervised Part Discovery from Contrastive Reconstruction](https://mlsanity.com/paper/2111.06349/) (2021)
- [Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery](https://mlsanity.com/paper/2105.01047/) (2021)
- [The Emergence of Objectness: Learning Zero-Shot Segmentation from Videos](https://mlsanity.com/paper/2111.06394/) (2021)
- [VAEL: Bridging Variational Autoencoders and Probabilistic Logic Programming](https://mlsanity.com/paper/2202.04178/) (2022)
- [Capturing the objects of vision with neural networks](https://mlsanity.com/paper/2109.03351/) (2021)
- [PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation](https://mlsanity.com/paper/2301.00954/) (2023)
- [Unsupervised Co-part Segmentation through Assembly](https://mlsanity.com/paper/2106.05897/) (2021)
- [Mitigating the Effect of Incidental Correlations on Part-based Learning](https://mlsanity.com/paper/2310.00377/) (2023)
- [MS-RNN: A Flexible Multi-Scale Framework for Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2206.03010/) (2022)
- [Slot Structured World Models](https://mlsanity.com/paper/2402.03326/) (2024)
- [ModeRNN: Harnessing Spatiotemporal Mode Collapse in Unsupervised Predictive Learning](https://mlsanity.com/paper/2110.03882/) (2021)
- [Structure from Action: Learning Interactions for Articulated Object 3D Structure Discovery](https://mlsanity.com/paper/2207.08997/) (2022)
- [Neural Marionette: Unsupervised Learning of Motion Skeleton and Latent Dynamics from Volumetric Video](https://mlsanity.com/paper/2202.08418/) (2022)
- [Boxhead: A Dataset for Learning Hierarchical Representations](https://mlsanity.com/paper/2110.03628/) (2021)
- [TKN: Transformer-based Keypoint Prediction Network For Real-time Video Prediction](https://mlsanity.com/paper/2303.09807/) (2023)
- [Wide and Narrow: Video Prediction from Context and Motion](https://mlsanity.com/paper/2110.11586/) (2021)
- [APES: Articulated Part Extraction from Sprite Sheets](https://mlsanity.com/paper/2206.02015/) (2022)
- [MOSH-WM: Mask-Grounded Soft-Hamiltonian Dynamics for Object-Centric World Models](https://mlsanity.com/paper/2608.22750/) (2026)

## Abstract (arXiv)

Humans easily recognize object parts and their hierarchical structure by watching how they move; they can then predict how each part moves in the future. In this paper, we propose a novel formulation that simultaneously learns a hierarchical, disentangled object representation and a dynamics model for object parts from unlabeled videos. Our Parts, Structure, and Dynamics (PSD) model learns to, first, recognize the object parts via a layered image representation; second, predict hierarchy via a structural descriptor that composes low-level concepts into a hierarchical structure; and third, model the system dynamics by predicting the future. Experiments on multiple real and synthetic datasets demonstrate that our PSD model works well on all three tasks: segmenting object parts, building their hierarchical structure, and capturing their motion distributions.