# MonoNPHM: Dynamic Head Reconstruction from Monocular Videos

arXiv: https://arxiv.org/abs/2312.06740 · HTML: https://mlsanity.com/paper/2312.06740/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 72 of its 97 references that are in the map.
- First posted on arXiv 11 Dec 2023.

## Builds on (most cited first, 30 of 72)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion](https://mlsanity.com/paper/2212.06135/) (2022)
- [Mixture of Volumetric Primitives for Efficient Neural Rendering](https://mlsanity.com/paper/2103.01954/) (2021)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019)
- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [I M Avatar: Implicit Morphable Head Avatars from Videos](https://mlsanity.com/paper/2112.07471/) (2021)
- [HeadNeRF: A Real-time NeRF-based Parametric Head Model](https://mlsanity.com/paper/2112.05637/) (2021)
- [SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes](https://mlsanity.com/paper/2104.03953/) (2021)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)

## Built on by (most cited first, 30 of 35)

- [FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models](https://mlsanity.com/paper/2312.08459/) (2023)
- [CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models](https://mlsanity.com/paper/2412.12093/) (2024)
- [NPGA: Neural Parametric Gaussian Avatars](https://mlsanity.com/paper/2405.19331/) (2024)
- [HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priors](https://mlsanity.com/paper/2408.06019/) (2024)
- [Gaussian Eigen Models for Human Heads](https://mlsanity.com/paper/2407.04545/) (2024)
- [Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction](https://mlsanity.com/paper/2505.00615/) (2025)
- [Joker: Conditional 3D Head Synthesis with Extreme Facial Expressions](https://mlsanity.com/paper/2410.16395/) (2024)
- [InsTaG: Learning Personalized 3D Talking Head from Few-Second Video](https://mlsanity.com/paper/2502.20387/) (2025)
- [SurFhead: Affine Rig Blending for Geometrically Accurate 2D Gaussian Surfel Head Avatars](https://mlsanity.com/paper/2410.11682/) (2024)
- [ProbTalk3D: Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAE](https://mlsanity.com/paper/2409.07966/) (2024)
- [ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions](https://mlsanity.com/paper/2507.10542/) (2025)
- [GPHM: Gaussian Parametric Head Model for Monocular Head Avatar Reconstruction](https://mlsanity.com/paper/2407.15070/) (2024)
- [GASP: Gaussian Avatars with Synthetic Priors](https://mlsanity.com/paper/2412.07739/) (2024)
- [SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting](https://mlsanity.com/paper/2412.14371/) (2024)
- [ImHead: A Large-scale Implicit Morphable Model for Localized Head Modeling](https://mlsanity.com/paper/2510.10793/) (2025)
- [LSF-Animation: Label-Free Speech-Driven Facial Animation via Implicit Feature Representation](https://mlsanity.com/paper/2510.21864/) (2025)
- [SRM-Hair: Single Image Head Mesh Reconstruction via 3D Morphable Hair](https://mlsanity.com/paper/2503.06154/) (2025)
- [BFSM: 3D Bidirectional Face-Skull Morphable Model](https://mlsanity.com/paper/2509.24577/) (2025)
- [TexAvatars : Hybrid Texel-3D Representations for Stable Rigging of Photorealistic Gaussian Head Avatars](https://mlsanity.com/paper/2512.21099/) (2025)
- [LightHeadEd: Relightable & Editable Head Avatars from a Smartphone](https://mlsanity.com/paper/2504.09671/) (2025)
- [GTAvatar: Bridging Gaussian Splatting and Texture Mapping for Relightable and Editable Gaussian Avatars](https://mlsanity.com/paper/2512.09162/) (2025)
- [Pix2NPHM: Learning to Regress NPHM Reconstructions From a Single Image](https://mlsanity.com/paper/2512.17773/) (2025)
- [Large-Scale High-Quality 3D Gaussian Head Reconstruction from Multi-View Captures](https://mlsanity.com/paper/2605.04035/) (2026)
- [Multi-view Consistent 3D Gaussian Head Avatars 'without' Multi-view Generation](https://mlsanity.com/paper/2605.25220/) (2026)
- [FiCA: Feed-forward instant Gaussian Codec Avatars from a Single Portrait Image](https://mlsanity.com/paper/2606.24232/) (2026)
- [Fast Registration of Photorealistic Avatars for VR Facial Animation](https://mlsanity.com/paper/2401.11002/) (2024)
- [GLVD: Guided Learned Vertex Descent](https://mlsanity.com/paper/2510.06046/) (2025)
- [Learning Neural Parametric 3D Breast Shape Models for Metrical Surface Reconstruction From Monocular RGB Videos](https://mlsanity.com/paper/2510.13540/) (2025)
- [Densemarks: Learning Canonical Embeddings for Human Heads Images via Point Tracks](https://mlsanity.com/paper/2511.02830/) (2025)
- [Using Gaussian Splats to Create High-Fidelity Facial Geometry and Texture](https://mlsanity.com/paper/2512.16397/) (2025)

## Abstract (arXiv)

We present Monocular Neural Parametric Head Models (MonoNPHM) for dynamic 3D head reconstructions from monocular RGB videos. To this end, we propose a latent appearance space that parameterizes a texture field on top of a neural parametric model. We constrain predicted color values to be correlated with the underlying geometry such that gradients from RGB effectively influence latent geometry codes during inverse rendering. To increase the representational capacity of our expression space, we augment our backward deformation field with hyper-dimensions, thus improving color and geometry representation in topologically challenging expressions. Using MonoNPHM as a learned prior, we approach the task of 3D head reconstruction using signed distance field based volumetric rendering. By numerically inverting our backward deformation field, we incorporated a landmark loss using facial anchor points that are closely tied to our canonical geometry representation. To evaluate the task of dynamic face reconstruction from monocular RGB videos we record 20 challenging Kinect sequences under casual conditions. MonoNPHM outperforms all baselines with a significant margin, and makes an important step towards easily accessible neural parametric face models through RGB tracking.