# PAniC-3D: Stylized Single-view 3D Reconstruction from Portraits of Anime Characters

arXiv: https://arxiv.org/abs/2303.14587 · HTML: https://mlsanity.com/paper/2303.14587/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 25 Mar 2023.

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation](https://mlsanity.com/paper/2112.11427/) (2021)
- [U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation](https://mlsanity.com/paper/1907.10830/) (2019)
- [GIRAFFE HD: A High-Resolution 3D-aware Generative Model](https://mlsanity.com/paper/2203.14954/) (2022)
- [3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks](https://mlsanity.com/paper/1707.06375/) (2017)
- [Improving the Perceptual Quality of 2D Animation Interpolation](https://mlsanity.com/paper/2111.12792/) (2021)
- [Mastering Sketching: Adversarial Augmentation for Structured Prediction](https://mlsanity.com/paper/1703.08966/) (2017)
- [AnimeCeleb: Large-Scale Animation CelebHeads Dataset for Head Reenactment](https://mlsanity.com/paper/2111.07640/) (2021)
- [Transfer Learning for Pose Estimation of Illustrated Characters](https://mlsanity.com/paper/2108.01819/) (2021)
- [Collaborative Neural Rendering using Anime Character Sheets](https://mlsanity.com/paper/2207.05378/) (2022)

## Built on by (most cited first, 17 of 17)

- [SAMPart3D: Segment Any Part in 3D Objects](https://mlsanity.com/paper/2411.07184/) (2024)
- [CharacterGen: Efficient 3D Character Generation from Single Images with Multi-View Pose Canonicalization](https://mlsanity.com/paper/2402.17214/) (2024)
- [Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Plane](https://mlsanity.com/paper/2403.16210/) (2024)
- [Real3D: Scaling Up Large Reconstruction Models with Real-World Images](https://mlsanity.com/paper/2406.08479/) (2024)
- [APISR: Anime Production Inspired Real-World Anime Super-Resolution](https://mlsanity.com/paper/2403.01598/) (2024)
- [Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review](https://mlsanity.com/paper/2402.11141/) (2024)
- [CharacterShot: Controllable and Consistent 4D Character Animation](https://mlsanity.com/paper/2508.07409/) (2025)
- [One missing piece in Vision and Language: A Survey on Comics Understanding](https://mlsanity.com/paper/2409.09502/) (2024)
- [ResGS: Residual Densification of 3D Gaussian for Efficient Detail Recovery](https://mlsanity.com/paper/2412.07494/) (2024)
- [SeqTex: Generate Mesh Textures in Video Sequence](https://mlsanity.com/paper/2507.04285/) (2025)
- [Toon3D: Seeing Cartoons from New Perspectives](https://mlsanity.com/paper/2405.10320/) (2024)
- [See-through: Single-image Layer Decomposition for Anime Characters](https://mlsanity.com/paper/2602.03749/) (2026)
- [LinkTo-Anime: A 2D Animation Optical Flow Dataset from 3D Model Rendering](https://mlsanity.com/paper/2506.02733/) (2025)
- [DreamCharacter-1: From 3D Generative Foundation Models to Product-Ready Character Generation](https://mlsanity.com/paper/2607.07817/) (2026)
- [Talking Head(?) Anime from a Single Image 4: Improved Model and Its Distillation](https://mlsanity.com/paper/2311.17409/) (2023)
- [GarmentPainter: Efficient 3D Garment Texture Synthesis with Character-Guided Diffusion Model](https://mlsanity.com/paper/2603.08228/) (2026)
- [NOFACE3D: NOise-scheduled Focused Anchoring and Controlled Enhancement for 3D Generation](https://mlsanity.com/paper/2610.13050/) (2026)

## Abstract (arXiv)

We propose PAniC-3D, a system to reconstruct stylized 3D character heads directly from illustrated (p)ortraits of (ani)me (c)haracters. Our anime-style domain poses unique challenges to single-view reconstruction; compared to natural images of human heads, character portrait illustrations have hair and accessories with more complex and diverse geometry, and are shaded with non-photorealistic contour lines. In addition, there is a lack of both 3D model and portrait illustration data suitable to train and evaluate this ambiguous stylized reconstruction task. Facing these challenges, our proposed PAniC-3D architecture crosses the illustration-to-3D domain gap with a line-filling model, and represents sophisticated geometries with a volumetric radiance field. We train our system with two large new datasets (11.2k Vroid 3D models, 1k Vtuber portrait illustrations), and evaluate on a novel AnimeRecon benchmark of illustration-to-3D pairs. PAniC-3D significantly outperforms baseline methods, and provides data to establish the task of stylized reconstruction from portrait illustrations.