mlsanityData to 14 Oct 2026

RigNeRF: Fully Controllable Neural 3D Portraits

arXiv 2206.06481 ↗First posted 13 Jun 2022cs.CVcited by 126

Not on the map. Why?

Abstract (arXiv)

Volumetric neural rendering methods, such as neural radiance fields (NeRFs), have enabled photo-realistic novel view synthesis. However, in their standard form, NeRFs do not support the editing of objects, such as a human head, within a scene. In this work, we propose RigNeRF, a system that goes beyond just novel view synthesis and enables full control of head pose and facial expressions learned from a single portrait video. We model changes in head pose and facial expressions using a deformation field that is guided by a 3D morphable face model (3DMM). The 3DMM effectively acts as a prior for RigNeRF that learns to predict only residuals to the 3DMM deformations and allows us to render novel (rigid) poses and (non-rigid) expressions that were not present in the input sequence. Using only a smartphone-captured short video of a subject for training, we demonstrate the effectiveness of our method on free view synthesis of a portrait scene with explicit head pose and expression controls. The project page can be found here: http://shahrukhathar.github.io/2022/06/06/RigNeRF.html

Key facts. Cited by 126 papers in the map (data through 14 Oct 2026). Builds on 41 of its 53 references that are in the map. First posted on arXiv 13 Jun 2022.

Builds on

41 of 53 references.

References · 41

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  5. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  6. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,232
  7. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  8. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  9. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,039
  10. 2020Neural Sparse Voxel Fieldscited by 959
  11. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  12. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
29 more
  1. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 847
  2. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 751
  3. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  4. 2020First Order Motion Model for Image Animationcited by 704
  5. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 646
  6. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 477
  7. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  8. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  9. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  10. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  11. 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 268
  12. 2018Deep Video Portraitscited by 267
  13. 2019Deferred Neural Rendering: Image Synthesis using Neural Texturescited by 253
  14. 2020Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learningcited by 220
  15. 2020Towards Fast, Accurate and Stable 3D Dense Face Alignmentcited by 219
  16. 2021NeX: Real-time View Synthesis with Neural Basis Expansioncited by 216
  17. 2020Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Datacited by 201
  18. 2021Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenescited by 190
  19. 2020Stable View Synthesiscited by 165
  20. 2017Neural Face Editing with Intrinsic Image Disentanglingcited by 70
  21. 2020CONFIG: Controllable Neural Face Image Generationcited by 61
  22. 2020Portrait Neural Radiance Fields from a Single Imagecited by 61
  23. 2018Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearancecited by 58
  24. 2020PIE: Portrait Image Embedding for Semantic Controlcited by 41
  25. 2020Head2Head: Video-based Neural Head Synthesiscited by 40
  26. 2020Head2Head++: Deep Facial Attributes Re-Targetingcited by 30
  27. 2021SIDER: Single-Image Neural Optimization for Facial Geometric Detail Recoverycited by 9
  28. 2019Self-supervised Deformation Modeling for Facial Expression Editingcited by 7
  29. 2020FaceDet3D: Facial Expressions with 3D Geometric Detail Predictioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 15; 2023: 37; 2024: 38; 2025: 19; 2026: 17.

126 papers.

Sort:

By area:

  1. 2023NeRSemble: Multi-view Radiance Field Reconstruction of Human Headscited by 194
  2. 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 184
  3. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 179
  4. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generationcited by 162
  5. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 158
  6. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  7. 2023FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embeddingcited by 108
  8. 2024Generalizable and Animatable Gaussian Head Avatarcited by 97
  9. 2023MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatarcited by 89
  10. 2022BAD-NeRF: Bundle Adjusted Deblur Neural Radiance Fieldscited by 80
  11. 2024Fully Explicit Dynamic Gaussian Splattingcited by 76
  12. 2024GPAvatar: Generalizable and Precise Head Avatar from Image(s)cited by 74
38 more
  1. 2022AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxelscited by 73
  2. 2023One-Shot High-Fidelity Talking-Head Synthesis with Deformable Neural Radiance Fieldcited by 68
  3. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  4. 2023NOFA: NeRF-based One-shot Facial Avatar Reconstructioncited by 53
  5. 2023LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatarcited by 50
  6. 2024NPGA: Neural Parametric Gaussian Avatarscited by 46
  7. 2023Learning Personalized High Quality Volumetric Head Avatars from Monocular RGB Videoscited by 44
  8. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  9. 2024SplineGS: Robust Motion-Adaptive Spline for Real-Time Dynamic 3D Gaussians from Monocular Videocited by 44
  10. 2024URAvatar: Universal Relightable Gaussian Codec Avatarscited by 43
  11. 2025LAM: Large Avatar Model for One-shot Animatable Gaussian Headcited by 43
  12. 2023DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatarscited by 42
  13. 2023TriPlaneNet: An Encoder for EG3D Inversioncited by 41
  14. 2023HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Fieldcited by 37
  15. 2022VolTeMorph: Realtime, Controllable and Generalisable Animation of Volumetric Representationscited by 36
  16. 2023HHAvatar: Gaussian Head Avatar with Dynamic Hairscited by 35
  17. 2023MonoNPHM: Dynamic Head Reconstruction from Monocular Videoscited by 35
  18. 2023VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactmentcited by 33
  19. 2023CoGS: Controllable Gaussian Splattingcited by 32
  20. 2024GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representationscited by 31
  21. 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
  22. 2023AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatarscited by 30
  23. 2025RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatarscited by 29
  24. 2022High-fidelity Facial Avatar Reconstruction from Monocular Video with Generative Priorscited by 28
  25. 2023CVTHead: One-shot Controllable Head Avatar with Vertex-feature Transformercited by 24
  26. 20223DFaceShop: Explicitly Controllable 3D-Aware Portrait Generationcited by 22
  27. 2025GUAVA: Generalizable Upper Body 3D Gaussian Avatarcited by 22
  28. 2023Generalizable One-shot Neural Head Avatarcited by 21
  29. 2023InvertAvatar: Incremental GAN Inversion for Generalized Head Avatarscited by 19
  30. 2024GaussianTalker: Real-Time High-Fidelity Talking Head Synthesis with Audio-Driven 3D Gaussian Splattingcited by 17
  31. 2023Single-Shot Implicit Morphable Faces with Consistent Texture Parameterizationcited by 16
  32. 2024Tri2-plane: Thinking Head Avatar via Feature Pyramidcited by 16
  33. 2023Template-free Articulated Neural Point Clouds for Reposable View Synthesiscited by 15
  34. 2024SurFhead: Affine Rig Blending for Geometrically Accurate 2D Gaussian Surfel Head Avatarscited by 15
  35. 2023Implicit Neural Head Synthesis via Controllable Local Deformation Fieldscited by 14
  36. 2024One2Avatar: Generative Implicit Head Avatar For Few-shot User Adaptationcited by 14
  37. 2024UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with Authenticity Guided Texturescited by 13
  38. 2023Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Imagescited by 12

50 most cited of 126 All 126 (JSON)