# Collaborative Neural Rendering using Anime Character Sheets

arXiv: https://arxiv.org/abs/2207.05378 · HTML: https://mlsanity.com/paper/2207.05378/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 75 references that are in the map.
- First posted on arXiv 12 Jul 2022.

## Builds on (most cited first, 30 of 48)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)
- [Motion Representations for Articulated Animation](https://mlsanity.com/paper/2104.11280/) (2021)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [View Synthesis by Appearance Flow](https://mlsanity.com/paper/1605.03557/) (2016)
- [Tex2Shape: Detailed Full Human Body Geometry From a Single Image](https://mlsanity.com/paper/1904.08645/) (2019)
- [Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis](https://mlsanity.com/paper/1909.12224/) (2019)
- [Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing](https://mlsanity.com/paper/2108.08807/) (2021)
- [Single Image Reflection Separation with Perceptual Losses](https://mlsanity.com/paper/1806.05376/) (2018)
- [Synthesizing Images of Humans in Unseen Poses](https://mlsanity.com/paper/1804.07739/) (2018)
- [Continuous Surface Embeddings](https://mlsanity.com/paper/2011.12438/) (2020)
- [Learning to Paint With Model-based Deep Reinforcement Learning](https://mlsanity.com/paper/1903.04411/) (2019)

## Built on by (most cited first, 8 of 8)

- [ControlNeXt: Powerful and Efficient Control for Image and Video Generation](https://mlsanity.com/paper/2408.06070/) (2024)
- [Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes](https://mlsanity.com/paper/2303.02760/) (2023)
- [PAniC-3D: Stylized Single-view 3D Reconstruction from Portraits of Anime Characters](https://mlsanity.com/paper/2303.14587/) (2023)
- [Generative AI for Cel-Animation: A Survey](https://mlsanity.com/paper/2501.06250/) (2025)
- [One missing piece in Vision and Language: A Survey on Comics Understanding](https://mlsanity.com/paper/2409.09502/) (2024)
- [ViStoryBench: Comprehensive Benchmark Suite for Story Visualization](https://mlsanity.com/paper/2505.24862/) (2025)
- [Talking Head(?) Anime from a Single Image 4: Improved Model and Its Distillation](https://mlsanity.com/paper/2311.17409/) (2023)
- [Learning High-Fidelity Cloth Animation via Skinning-Free Image Transfer](https://mlsanity.com/paper/2512.05593/) (2025)

## Abstract (arXiv)

Drawing images of characters with desired poses is an essential but laborious task in anime production. Assisting artists to create is a research hotspot in recent years. In this paper, we present the Collaborative Neural Rendering (CoNR) method, which creates new images for specified poses from a few reference images (AKA Character Sheets). In general, the diverse hairstyles and garments of anime characters defies the employment of universal body models like SMPL, which fits in most nude human shapes. To overcome this, CoNR uses a compact and easy-to-obtain landmark encoding to avoid creating a unified UV mapping in the pipeline. In addition, the performance of CoNR can be significantly improved when referring to multiple reference images, thanks to feature space cross-view warping in a carefully designed neural network. Moreover, we have collected a character sheet dataset containing over 700,000 hand-drawn and synthesized images of diverse poses to facilitate research in this area. Our code and demo are available at https://github.com/megvii-research/IJCAI2023-CoNR.