mlsanityData to 14 Oct 2026

MonoNPHM: Dynamic Head Reconstruction from Monocular Videos

arXiv 2312.06740 ↗First posted 11 Dec 2023cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

We present Monocular Neural Parametric Head Models (MonoNPHM) for dynamic 3D head reconstructions from monocular RGB videos. To this end, we propose a latent appearance space that parameterizes a texture field on top of a neural parametric model. We constrain predicted color values to be correlated with the underlying geometry such that gradients from RGB effectively influence latent geometry codes during inverse rendering. To increase the representational capacity of our expression space, we augment our backward deformation field with hyper-dimensions, thus improving color and geometry representation in topologically challenging expressions. Using MonoNPHM as a learned prior, we approach the task of 3D head reconstruction using signed distance field based volumetric rendering. By numerically inverting our backward deformation field, we incorporated a landmark loss using facial anchor points that are closely tied to our canonical geometry representation. To evaluate the task of dynamic face reconstruction from monocular RGB videos we record 20 challenging Kinect sequences under casual conditions. MonoNPHM outperforms all baselines with a significant margin, and makes an important step towards easily accessible neural parametric face models through RGB tracking.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 72 of its 97 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

72 of 97 references.

References · 72

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  7. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  9. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  10. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  11. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  12. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
38 more
  1. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  2. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  3. 2020Convolutional Occupancy Networkscited by 761
  4. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  5. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  6. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 632
  7. 2021Neural Fields in Visual Computing and Beyondcited by 540
  8. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 477
  9. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  10. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  11. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  12. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 268
  13. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 260
  14. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 255
  15. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 231
  16. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  17. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  18. 2021Neural Head Avatars from Monocular RGB Videoscited by 213
  19. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  20. 2021General Facial Representation Learning in a Visual-Linguistic Mannercited by 195
  21. 2023NeRSemble: Multi-view Radiance Field Reconstruction of Human Headscited by 194
  22. 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 184
  23. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  24. 2023HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Samplingcited by 174
  25. 2022Towards Metrical Reconstruction of Human Facescited by 166
  26. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  27. 2023HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motioncited by 161
  28. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
  29. 2020MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decompositioncited by 127
  30. 2022RigNeRF: Fully Controllable Neural 3D Portraitscited by 126
  31. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  32. 2022Generative Neural Articulated Radiance Fieldscited by 101
  33. 2022FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Datasetcited by 98
  34. 2021MoFaNeRF: Morphable Facial Neural Radiance Fieldcited by 94
  35. 2023PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360circcited by 93
  36. 2021NPMs: Neural Parametric Models for 3D Deformable Shapescited by 90
  37. 2019SpiralNet++: A Fast and Highly Efficient Mesh Convolution Operatorcited by 89
  38. 2021H3D-Net: Few-Shot High-Fidelity 3D Head Reconstructioncited by 81

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 11; 2025: 16; 2026: 7.

35 papers.

Sort:

By area:

  1. 2023FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Modelscited by 55
  2. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  3. 2024NPGA: Neural Parametric Gaussian Avatarscited by 46
  4. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  5. 2024Gaussian Eigen Models for Human Headscited by 21
  6. 2025Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstructioncited by 20
  7. 2024Joker: Conditional 3D Head Synthesis with Extreme Facial Expressionscited by 18
  8. 2025InsTaG: Learning Personalized 3D Talking Head from Few-Second Videocited by 18
  9. 2024SurFhead: Affine Rig Blending for Geometrically Accurate 2D Gaussian Surfel Head Avatarscited by 15
  10. 2024ProbTalk3D: Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAEcited by 13
  11. 2025ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressionscited by 12
  12. 2024GPHM: Gaussian Parametric Head Model for Monocular Head Avatar Reconstructioncited by 10
23 more
  1. 2024GASP: Gaussian Avatars with Synthetic Priorscited by 7
  2. 2024SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargetingcited by 4
  3. 2025ImHead: A Large-scale Implicit Morphable Model for Localized Head Modelingcited by 4
  4. 2025LSF-Animation: Label-Free Speech-Driven Facial Animation via Implicit Feature Representationcited by 3
  5. 2025SRM-Hair: Single Image Head Mesh Reconstruction via 3D Morphable Haircited by 2
  6. 2025BFSM: 3D Bidirectional Face-Skull Morphable Modelcited by 2
  7. 2025TexAvatars : Hybrid Texel-3D Representations for Stable Rigging of Photorealistic Gaussian Head Avatarscited by 2
  8. 2025LightHeadEd: Relightable & Editable Head Avatars from a Smartphonecited by 1
  9. 2025GTAvatar: Bridging Gaussian Splatting and Texture Mapping for Relightable and Editable Gaussian Avatarscited by 1
  10. 2025Pix2NPHM: Learning to Regress NPHM Reconstructions From a Single Imagecited by 1
  11. 2026Large-Scale High-Quality 3D Gaussian Head Reconstruction from Multi-View Capturescited by 1
  12. 2026Multi-view Consistent 3D Gaussian Head Avatars 'without' Multi-view Generationcited by 1
  13. 2026FiCA: Feed-forward instant Gaussian Codec Avatars from a Single Portrait Imagecited by 1
  14. 2024Fast Registration of Photorealistic Avatars for VR Facial Animationcited by 0
  15. 2025GLVD: Guided Learned Vertex Descentcited by 0
  16. 2025Learning Neural Parametric 3D Breast Shape Models for Metrical Surface Reconstruction From Monocular RGB Videoscited by 0
  17. 2025Densemarks: Learning Canonical Embeddings for Human Heads Images via Point Trackscited by 0
  18. 2025Using Gaussian Splats to Create High-Fidelity Facial Geometry and Texturecited by 0
  19. 2025Instant Expressive Gaussian Head Avatars at Over 100 FPScited by 0
  20. 2026GaussianSwap: Animatable Video Face Swapping with 3D Gaussian Splattingcited by 0
  21. 2026Self-Learning Expression Deformations for Data-Efficient Gaussian Avatarscited by 0
  22. 2026FFAvatar: Feed-Forward 4D Head Avatar Reconstruction from Sparse Portrait Imagescited by 0
  23. 2026ViDS: Video Diffusion Shader using 3D Face Trackingcited by 0