mlsanityData to 7 Oct 2026

Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video

arXiv 2210.06108 ↗First posted 12 Oct 2022cs.GR cs.CVcited by 128

Not on the map. Why?

Abstract (arXiv)

We present a novel semantic model for human head defined with neural radiance field. The 3D-consistent head model consist of a set of disentangled and interpretable bases, and can be driven by low-dimensional expression coefficients. Thanks to the powerful representation ability of neural radiance field, the constructed model can represent complex facial attributes including hair, wearings, which can not be represented by traditional mesh blendshape. To construct the personalized semantic facial model, we propose to define the bases as several multi-level voxel fields. With a short monocular RGB video as input, our method can construct the subject's semantic facial NeRF model with only ten to twenty minutes, and can render a photo-realistic human head image in tens of miliseconds with a given expression coefficient and view direction. With this novel representation, we apply it to many tasks like facial retargeting and expression editing. Experimental results demonstrate its strong representation ability and training/inference speed. Demo videos and released code are provided in our project page: https://ustc3dv.github.io/NeRFBlendShape/

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). Builds on 54 of its 74 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

54 of 74 references.

References · 54

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  5. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  6. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  7. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  8. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  9. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  10. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  11. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  12. 2021Volume Rendering of Neural Implicit Surfacescited by 990
38 more
  1. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  2. 2020Neural Sparse Voxel Fieldscited by 959
  3. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  4. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  5. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  6. 2020First Order Motion Model for Image Animationcited by 698
  7. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  8. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  9. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  10. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  11. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  12. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  13. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  14. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  15. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  16. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  17. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  18. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  19. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 267
  20. 2018Deep Video Portraitscited by 265
  21. 2019Deferred Neural Rendering: Image Synthesis using Neural Texturescited by 253
  22. 2019Neural Voice Puppetry: Audio-driven Facial Reenactmentcited by 249
  23. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 230
  24. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  25. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  26. 2022NeRF-Editing: Geometry Editing of Neural Radiance Fieldscited by 216
  27. 2021Neural Head Avatars from Monocular RGB Videoscited by 212
  28. 2021HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fieldscited by 210
  29. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  30. 2021Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenescited by 190
  31. 2018Nonlinear 3D Face Morphable Modelcited by 157
  32. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  33. 2022SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Videocited by 144
  34. 2022NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editingcited by 141
  35. 2022Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generationcited by 119
  36. 2020Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatarscited by 116
  37. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  38. 2021MoFaNeRF: Morphable Facial Neural Radiance Fieldcited by 94

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 38; 2024: 42; 2025: 22; 2026: 20.

128 papers.

Sort:

By area:

  1. 2023GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussianscited by 256
  2. 2023NeRSemble: Multi-view Radiance Field Reconstruction of Human Headscited by 190
  3. 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 183
  4. 2022Instant Volumetric Head Avatarscited by 157
  5. 2023FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embeddingcited by 107
  6. 2024Generalizable and Animatable Gaussian Head Avatarcited by 95
  7. 2023MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatarcited by 89
  8. 2023AvatarReX: Real-time Expressive Full-body Avatarscited by 83
  9. 2022Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decompositioncited by 81
  10. 2022AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxelscited by 73
  11. 2023Learning Neural Volumetric Representations of Dynamic Humans in Minutescited by 70
  12. 2024Advances in 3D Generation: A Surveycited by 62
38 more
  1. 2022Interactive Segmentation of Radiance Fieldscited by 60
  2. 2023HeadGaS: Real-Time Animatable Head Avatars via 3D Gaussian Splattingcited by 60
  3. 20243D Gaussian Blendshapes for Head Avatar Animationcited by 58
  4. 2023NOFA: NeRF-based One-shot Facial Avatar Reconstructioncited by 53
  5. 2023StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Videocited by 52
  6. 2023PoseVocab: Learning Joint-structured Pose Embeddings for Human Avatar Modelingcited by 50
  7. 2023LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatarcited by 50
  8. 2023GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivationcited by 46
  9. 2023RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatarscited by 45
  10. 2023Learning Disentangled Avatars with Hybrid 3D Representationscited by 44
  11. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  12. 2025LAM: Large Avatar Model for One-shot Animatable Gaussian Headcited by 43
  13. 2023Text-Guided Generation and Editing of Compositional 3D Avatarscited by 39
  14. 2023FLARE: Fast Learning of Animatable and Relightable Mesh Avatarscited by 39
  15. 2024MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editingcited by 37
  16. 2023Efficient Large-scale Scene Representation with a Hybrid of High-resolution Grid and Plane Featurescited by 36
  17. 2023DELIFFAS: Deformable Light Fields for Fast Avatar Synthesiscited by 36
  18. 2023HHAvatar: Gaussian Head Avatar with Dynamic Hairscited by 35
  19. 2023Towards Practical Capture of High-Fidelity Relightable Avatarscited by 34
  20. 2024GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representationscited by 31
  21. 2024FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Videocited by 31
  22. 2023AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatarscited by 29
  23. 2022High-fidelity Facial Avatar Reconstruction from Monocular Video with Generative Priorscited by 28
  24. 2024MegActor: Harness the Power of Raw Video for Vivid Portrait Animationcited by 27
  25. 2025RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatarscited by 27
  26. 2023Generalizable One-shot Neural Head Avatarcited by 21
  27. 2024Gaussian Eigen Models for Human Headscited by 21
  28. 2023BakedAvatar: Baking Neural Fields for Real-Time Head Avatar Synthesiscited by 20
  29. 2024VRMM: A Volumetric Relightable Morphable Head Modelcited by 19
  30. 2024VOODOO XP: Expressive One-Shot Head Reenactment for VR Telepresencecited by 19
  31. 2024PSAvatar: A Point-based Shape Model for Real-Time Head Avatar Animation with 3D Gaussian Splattingcited by 17
  32. 2024GaussianTalker: Real-Time High-Fidelity Talking Head Synthesis with Audio-Driven 3D Gaussian Splattingcited by 17
  33. 2024Tri2-plane: Thinking Head Avatar via Feature Pyramidcited by 16
  34. 2024SurFhead: Affine Rig Blending for Geometrically Accurate 2D Gaussian Surfel Head Avatarscited by 15
  35. 2023Implicit Neural Head Synthesis via Controllable Local Deformation Fieldscited by 14
  36. 2025Synthetic Prior for Few-Shot Drivable Head Avatar Inversioncited by 14
  37. 2024UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with Authenticity Guided Texturescited by 13
  38. 2024A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generationcited by 13

50 most cited of 128 All 128 (JSON)