# High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies

arXiv: https://arxiv.org/abs/2010.05562 · HTML: https://mlsanity.com/paper/2010.05562/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2020.

## Built on by (most cited first, 30 of 47)

- [FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset](https://mlsanity.com/paper/2203.14057/) (2022)
- [FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction](https://mlsanity.com/paper/2211.13874/) (2022)
- [FitMe: Deep Photorealistic 3D Morphable Model Avatars](https://mlsanity.com/paper/2305.09641/) (2023)
- [MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing](https://mlsanity.com/paper/2404.19026/) (2024)
- [MonoNPHM: Dynamic Head Reconstruction from Monocular Videos](https://mlsanity.com/paper/2312.06740/) (2023)
- [Relightify: Relightable 3D Faces from a Single Image via Diffusion Models](https://mlsanity.com/paper/2305.06077/) (2023)
- [REALY: Rethinking the Evaluation of 3D Face Reconstruction](https://mlsanity.com/paper/2203.09729/) (2022)
- [NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animation](https://mlsanity.com/paper/2211.17235/) (2022)
- [HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details](https://mlsanity.com/paper/2303.11225/) (2023)
- [TextToon: Real-Time Text Toonify Head Avatar from Single Video](https://mlsanity.com/paper/2410.07160/) (2024)
- [Video-driven Neural Physically-based Facial Asset for Production](https://mlsanity.com/paper/2202.05592/) (2022)
- [GANHead: Towards Generative Animatable Neural Head Avatars](https://mlsanity.com/paper/2304.03950/) (2023)
- [Neural Parameterization for Dynamic Human Head Editing](https://mlsanity.com/paper/2207.00210/) (2022)
- [ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performer](https://mlsanity.com/paper/2408.03284/) (2024)
- [Learning Audio-Driven Viseme Dynamics for 3D Face Animation](https://mlsanity.com/paper/2301.06059/) (2023)
- [Cafe-Talk: Generating 3D Talking Face Animation with Multimodal Coarse- and Fine-grained Control](https://mlsanity.com/paper/2503.14517/) (2025)
- [BareSkinNet: De-makeup and De-lighting via 3D Face Reconstruction](https://mlsanity.com/paper/2209.09029/) (2022)
- [Text-Guided 3D Face Synthesis -- From Generation to Editing](https://mlsanity.com/paper/2312.00375/) (2023)
- [Makeup Extraction of 3D Representation via Illumination-Aware Image Decomposition](https://mlsanity.com/paper/2302.13279/) (2023)
- [Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction](https://mlsanity.com/paper/2310.08784/) (2023)
- [Topo4D: Topology-Preserving Gaussian Splatting for High-Fidelity 4D Head Capture](https://mlsanity.com/paper/2406.00440/) (2024)
- [Makeup Prior Models for 3D Facial Makeup Estimation and Applications](https://mlsanity.com/paper/2403.17761/) (2024)
- [FreeUV: Ground-Truth-Free Realistic Facial UV Texture Recovery via Cross-Assembly Inference Strategy](https://mlsanity.com/paper/2503.17197/) (2025)
- [Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions](https://mlsanity.com/paper/2209.04436/) (2022)
- [A Perceptual Shape Loss for Monocular 3D Face Reconstruction](https://mlsanity.com/paper/2310.19580/) (2023)
- [Facial Appearance Capture at Home with Patch-Level Reflectance Prior](https://mlsanity.com/paper/2506.03478/) (2025)
- [Novel View Synthesis for High-fidelity Headshot Scenes](https://mlsanity.com/paper/2205.15595/) (2022)
- [SVG-Head: Hybrid Surface-Volumetric Gaussians for High-Fidelity Head Reconstruction and Real-Time Editing](https://mlsanity.com/paper/2508.09597/) (2025)
- [SCULPTOR: Skeleton-Consistent Face Creation Using a Learned Parametric Generator](https://mlsanity.com/paper/2209.06423/) (2022)
- [EmoHead: Emotional Talking Head via Manipulating Semantic Expression Parameters](https://mlsanity.com/paper/2503.19416/) (2025)

## Abstract (arXiv)

We present a fully automatic system that can produce high-fidelity, photo-realistic 3D digital human heads with a consumer RGB-D selfie camera. The system only needs the user to take a short selfie RGB-D video while rotating his/her head, and can produce a high quality head reconstruction in less than 30 seconds. Our main contribution is a new facial geometry modeling and reflectance synthesis procedure that significantly improves the state-of-the-art. Specifically, given the input video a two-stage frame selection procedure is first employed to select a few high-quality frames for reconstruction. Then a differentiable renderer based 3D Morphable Model (3DMM) fitting algorithm is applied to recover facial geometries from multiview RGB-D data, which takes advantages of a powerful 3DMM basis constructed with extensive data generation and perturbation. Our 3DMM has much larger expressive capacities than conventional 3DMM, allowing us to recover more accurate facial geometry using merely linear basis. For reflectance synthesis, we present a hybrid approach that combines parametric fitting and CNNs to synthesize high-resolution albedo/normal maps with realistic hair/pore/wrinkle details. Results show that our system can produce faithful 3D digital human faces with extremely realistic details. The main code and the newly constructed 3DMM basis is publicly available.