# APES: Articulated Part Extraction from Sprite Sheets

arXiv: https://arxiv.org/abs/2206.02015 · HTML: https://mlsanity.com/paper/2206.02015/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 62 references that are in the map.
- First posted on arXiv 4 Jun 2022.

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [COTR: Correspondence Transformer for Matching Across Images](https://mlsanity.com/paper/2103.14167/) (2021)
- [Shape2Motion: Joint Analysis of Motion Parts and Attributes from 3D Shapes](https://mlsanity.com/paper/1903.03911/) (2019)
- [SCOPS: Self-Supervised Co-Part Segmentation](https://mlsanity.com/paper/1905.01298/) (2019)
- [MIPaaL: Mixed Integer Program as a Layer](https://mlsanity.com/paper/1907.05912/) (2019)
- [Interior Point Solving for LP-based prediction+optimisation](https://mlsanity.com/paper/2010.13943/) (2020)
- [Continuous Surface Embeddings](https://mlsanity.com/paper/2011.12438/) (2020)
- [Deep Part Induction from Articulated Object Pairs](https://mlsanity.com/paper/1809.07417/) (2018)
- [Deep Feature Factorization For Concept Discovery](https://mlsanity.com/paper/1806.10206/) (2018)
- [Unsupervised Part Discovery from Contrastive Reconstruction](https://mlsanity.com/paper/2111.06349/) (2021)
- [Learning Motion Patterns in Videos](https://mlsanity.com/paper/1612.07217/) (2016)
- [Layered Neural Rendering for Retiming People in Video](https://mlsanity.com/paper/2009.07833/) (2020)
- [Learning to Segment Rigid Motions from Two Frames](https://mlsanity.com/paper/2101.03694/) (2021)
- [MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization](https://mlsanity.com/paper/2101.06605/) (2021)
- [Learning to Segment Moving Objects](https://mlsanity.com/paper/1712.01127/) (2017)
- [Motion-supervised Co-Part Segmentation](https://mlsanity.com/paper/2004.03234/) (2020)
- [Unsupervised part representation by Flow Capsules](https://mlsanity.com/paper/2011.13920/) (2020)
- [Unsupervised Discovery of Parts, Structure, and Dynamics](https://mlsanity.com/paper/1903.05136/) (2019)
- [CycleSegNet: Object Co-segmentation with Cycle Refinement and Region Correspondence](https://mlsanity.com/paper/2101.01308/) (2021)
- [3D Rigid Motion Segmentation with Mixed and Unknown Number of Models](https://mlsanity.com/paper/1908.06087/) (2019)
- [CharacterGAN: Few-Shot Keypoint Character Animation and Reposing](https://mlsanity.com/paper/2102.03141/) (2021)
- [Neural Puppet: Generative Layered Cartoon Characters](https://mlsanity.com/paper/1910.02060/) (2019)

## Abstract (arXiv)

Rigged puppets are one of the most prevalent representations to create 2D character animations. Creating these puppets requires partitioning characters into independently moving parts. In this work, we present a method to automatically identify such articulated parts from a small set of character poses shown in a sprite sheet, which is an illustration of the character that artists often draw before puppet creation. Our method is trained to infer articulated parts, e.g. head, torso and limbs, that can be re-assembled to best reconstruct the given poses. Our results demonstrate significantly better performance than alternatives qualitatively and quantitatively.Our project page https://zhan-xu.github.io/parts/ includes our code and data.