# i3DMM: Deep Implicit 3D Morphable Model of Human Heads

arXiv: https://arxiv.org/abs/2011.14143 · HTML: https://mlsanity.com/paper/2011.14143/

## Key facts

- Cited by 109 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2020.

## Built on by (most cited first, 30 of 109)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [I M Avatar: Implicit Morphable Head Avatars from Videos](https://mlsanity.com/paper/2112.07471/) (2021)
- [NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads](https://mlsanity.com/paper/2305.03027/) (2023)
- [Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video](https://mlsanity.com/paper/2210.06108/) (2022)
- [Generalizable Patch-Based Neural Rendering](https://mlsanity.com/paper/2207.10662/) (2022)
- [Realistic One-shot Mesh-based Head Avatars](https://mlsanity.com/paper/2206.08343/) (2022)
- [Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing](https://mlsanity.com/paper/2108.08807/) (2021)
- [Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis](https://mlsanity.com/paper/2207.11770/) (2022)
- [MoFaNeRF: Morphable Facial Neural Radiance Field](https://mlsanity.com/paper/2112.02308/) (2021)
- [AvatarReX: Real-time Expressive Full-body Avatars](https://mlsanity.com/paper/2305.04789/) (2023)
- [SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesis](https://mlsanity.com/paper/2404.04104/) (2024)
- [H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction](https://mlsanity.com/paper/2107.12512/) (2021)
- [Learning Neural Parametric Head Models](https://mlsanity.com/paper/2212.02761/) (2022)
- [A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis](https://mlsanity.com/paper/2110.15678/) (2021)
- [X-Avatar: Expressive Human Avatars](https://mlsanity.com/paper/2303.04805/) (2023)
- [ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2203.14510/) (2022)
- [LISA: Learning Implicit Shape and Appearance of Hands](https://mlsanity.com/paper/2204.01695/) (2022)
- [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024)
- [3D Gaussian Blendshapes for Head Avatar Animation](https://mlsanity.com/paper/2404.19398/) (2024)
- [CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphism](https://mlsanity.com/paper/2203.16529/) (2022)
- [LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatar](https://mlsanity.com/paper/2305.01190/) (2023)
- [NPGA: Neural Parametric Gaussian Avatars](https://mlsanity.com/paper/2405.19331/) (2024)
- [RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars](https://mlsanity.com/paper/2305.13353/) (2023)
- [Learning Disentangled Avatars with Hybrid 3D Representations](https://mlsanity.com/paper/2309.06441/) (2023)
- [AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis](https://mlsanity.com/paper/2204.03105/) (2022)
- [Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance](https://mlsanity.com/paper/2401.15687/) (2024)
- [Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Video](https://mlsanity.com/paper/2211.12782/) (2022)
- [HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Field](https://mlsanity.com/paper/2309.17128/) (2023)
- [FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction](https://mlsanity.com/paper/2111.01082/) (2021)

## Abstract (arXiv)

We present the first deep implicit 3D morphable model (i3DMM) of full heads. Unlike earlier morphable face models it not only captures identity-specific geometry, texture, and expressions of the frontal face, but also models the entire head, including hair. We collect a new dataset consisting of 64 people with different expressions and hairstyles to train i3DMM. Our approach has the following favorable properties: (i) It is the first full head morphable model that includes hair. (ii) In contrast to mesh-based models it can be trained on merely rigidly aligned scans, without requiring difficult non-rigid registration. (iii) We design a novel architecture to decouple the shape model into an implicit reference shape and a deformation of this reference shape. With that, dense correspondences between shapes can be learned implicitly. (iv) This architecture allows us to semantically disentangle the geometry and color components, as color is learned in the reference space. Geometry is further disentangled as identity, expressions, and hairstyle, while color is disentangled as identity and hairstyle components. We show the merits of i3DMM using ablation studies, comparisons to state-of-the-art models, and applications such as semantic head editing and texture transfer. We will make our model publicly available.