# Neural Puppet: Generative Layered Cartoon Characters

arXiv: https://arxiv.org/abs/1910.02060 · HTML: https://mlsanity.com/paper/1910.02060/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Oct 2019.

## Built on by (most cited first, 7 of 7)

- [Breathing Life Into Sketches Using Text-to-Video Priors](https://mlsanity.com/paper/2311.13608/) (2023)
- [A Method for Animating Children's Drawings of the Human Figure](https://mlsanity.com/paper/2303.12741/) (2023)
- [Transfer Learning for Pose Estimation of Illustrated Characters](https://mlsanity.com/paper/2108.01819/) (2021)
- [CharacterGAN: Few-Shot Keypoint Character Animation and Reposing](https://mlsanity.com/paper/2102.03141/) (2021)
- [Enhancing Sketch Animation: Text-to-Video Diffusion Models with Temporal Consistency and Rigidity Constraints](https://mlsanity.com/paper/2411.19381/) (2024)
- [See-through: Single-image Layer Decomposition for Anime Characters](https://mlsanity.com/paper/2602.03749/) (2026)
- [APES: Articulated Part Extraction from Sprite Sheets](https://mlsanity.com/paper/2206.02015/) (2022)

## Abstract (arXiv)

We propose a learning based method for generating new animations of a cartoon character given a few example images. Our method is designed to learn from a traditionally animated sequence, where each frame is drawn by an artist, and thus the input images lack any common structure, correspondences, or labels. We express pose changes as a deformation of a layered 2.5D template mesh, and devise a novel architecture that learns to predict mesh deformations matching the template to a target image. This enables us to extract a common low-dimensional structure from a diverse set of character poses. We combine recent advances in differentiable rendering as well as mesh-aware models to successfully align common template even if only a few character images are available during training. In addition to coarse poses, character appearance also varies due to shading, out-of-plane motions, and artistic effects. We capture these subtle changes by applying an image translation network to refine the mesh rendering, providing an end-to-end model to generate new animations of a character with high visual quality. We demonstrate that our generative model can be used to synthesize in-between frames and to create data-driven deformation. Our template fitting procedure outperforms state-of-the-art generic techniques for detecting image correspondences.