mlsanityData to 7 Oct 2026

Neural Head Avatars from Monocular RGB Videos

arXiv 2112.01554 ↗First posted 2 Dec 2021cs.CV cs.GRcited by 212

Not on the map. Why?

Abstract (arXiv)

We present Neural Head Avatars, a novel neural representation that explicitly models the surface geometry and appearance of an animatable human avatar that can be used for teleconferencing in AR/VR or other applications in the movie or games industry that rely on a digital human. Our representation can be learned from a monocular RGB portrait video that features a range of different expressions and views. Specifically, we propose a hybrid representation consisting of a morphable model for the coarse shape and expressions of the face, and two feed-forward networks, predicting vertex offsets of the underlying mesh as well as a view- and expression-dependent texture. We demonstrate that this representation is able to accurately extrapolate to unseen poses and view points, and generates natural expressions while providing sharp texture details. Compared to previous works on head avatars, our method provides a disentangled shape and appearance model of the complete human head (including hair) that is compatible with the standard graphics pipeline. Moreover, it quantitatively and qualitatively outperforms current state of the art in terms of reconstruction quality and novel-view synthesis.

Key facts. Cited by 212 papers in the map (data through 7 Oct 2026). Builds on 57 of its 98 references that are in the map. First posted on arXiv 2 Dec 2021.

Builds on

57 of 98 references.

References · 57

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  6. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  7. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  8. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  9. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  10. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  11. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  12. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
38 more
  1. 2020First Order Motion Model for Image Animationcited by 698
  2. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  3. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  4. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  5. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
  6. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
  7. 2015A Neural Algorithm of Artistic Stylecited by 480
  8. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  9. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  10. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  11. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  12. 2019FSGAN: Subject Agnostic Face Swapping and Reenactmentcited by 319
  13. 2020State of the Art on Neural Renderingcited by 276
  14. 20193D Morphable Face Models -- Past, Present and Futurecited by 269
  15. 2018Deep Video Portraitscited by 265
  16. 2019Deferred Neural Rendering: Image Synthesis using Neural Texturescited by 253
  17. 2019Neural Voice Puppetry: Audio-driven Facial Reenactmentcited by 249
  18. 2021HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fieldscited by 210
  19. 2021Neural Articulated Radiance Fieldcited by 196
  20. 2018Deep Appearance Models for Face Renderingcited by 164
  21. 2021H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motioncited by 157
  22. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
  23. 2021Pixel Codec Avatarscited by 125
  24. 2020H3DNet: 3D Object Detection Using Hybrid Geometric Primitivescited by 123
  25. 2018FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Facescited by 118
  26. 2020Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatarscited by 116
  27. 2018Detailed Human Avatars from Monocular Videocited by 109
  28. 2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106
  29. 2021Neural Lumigraph Renderingcited by 105
  30. 2018ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detectioncited by 104
  31. 2021imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Posecited by 102
  32. 2020ID-Reveal: Identity-aware DeepFake Video Detectioncited by 97
  33. 2020AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"cited by 91
  34. 2021H3D-Net: Few-Shot High-Fidelity 3D Head Reconstructioncited by 81
  35. 2020Learning Compositional Radiance Fields of Dynamic Human Headscited by 78
  36. 2020SMPLpix: Neural Avatars from 3D Human Modelscited by 69
  37. 2020Detecting Deep-Fake Videos from Appearance and Behaviorcited by 66
  38. 2021Dynamic Surface Function Networks for Clothed Human Bodiescited by 44

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 26; 2023: 75; 2024: 42; 2025: 39; 2026: 29.

212 papers.

Sort:

By area:

  1. 2023GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussianscited by 256
  2. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 230
  3. 2023NeRSemble: Multi-view Radiance Field Reconstruction of Human Headscited by 190
  4. 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 183
  5. 2024SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splattingcited by 171
  6. 2022Towards Metrical Reconstruction of Human Facescited by 165
  7. 2022Instant Volumetric Head Avatarscited by 157
  8. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 156
  9. 2022Reconstructing Personalized Semantic Facial NeRF Models From Monocular Videocited by 128
  10. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  11. 2022Realistic One-shot Mesh-based Head Avatarscited by 119
  12. 2023FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embeddingcited by 107
38 more
  1. 2022DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenescited by 103
  2. 2023MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatarcited by 89
  3. 2022Capturing and Animation of Body and Clothing from Monocular Videocited by 83
  4. 2023AvatarReX: Real-time Expressive Full-body Avatarscited by 83
  5. 2022KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypointscited by 77
  6. 2023DiffusionRig: Learning Personalized Priors for Facial Appearance Editingcited by 74
  7. 2022AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxelscited by 73
  8. 2023One-Shot High-Fidelity Talking-Head Synthesis with Deformable Neural Radiance Fieldcited by 68
  9. 2024EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatarscited by 65
  10. 2024Advances in 3D Generation: A Surveycited by 62
  11. 2023HeadGaS: Real-Time Animatable Head Avatars via 3D Gaussian Splattingcited by 60
  12. 2022Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Cameracited by 57
  13. 2023NOFA: NeRF-based One-shot Facial Avatar Reconstructioncited by 53
  14. 2023StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Videocited by 52
  15. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  16. 2025Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatarscited by 52
  17. 2022Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videoscited by 50
  18. 2023LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatarcited by 50
  19. 2023RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatarscited by 45
  20. 2023Learning Personalized High Quality Volumetric Head Avatars from Monocular RGB Videoscited by 44
  21. 2023Learning Disentangled Avatars with Hybrid 3D Representationscited by 44
  22. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  23. 2023Neural Haircut: Prior-Guided Strand-Based Hair Reconstructioncited by 43
  24. 2023High-Fidelity and Freely Controllable Talking Head Video Generationcited by 42
  25. 2023DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatarscited by 42
  26. 2023Text-Guided Generation and Editing of Compositional 3D Avatarscited by 39
  27. 2022Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Videocited by 37
  28. 2023HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Fieldcited by 37
  29. 2024MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editingcited by 37
  30. 2022VolTeMorph: Realtime, Controllable and Generalisable Animation of Volumetric Representationscited by 36
  31. 2024GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusioncited by 36
  32. 2023HHAvatar: Gaussian Head Avatar with Dynamic Hairscited by 35
  33. 2023MonoNPHM: Dynamic Head Reconstruction from Monocular Videoscited by 35
  34. 2023VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactmentcited by 33
  35. 2024FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Videocited by 31
  36. 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
  37. 2022NeuWigs: A Neural Dynamic Model for Volumetric Hair Capture and Animationcited by 30
  38. 2022EditableNeRF: Editing Topologically Varying Neural Radiance Fields by Key Pointscited by 30

50 most cited of 212 All 212 (JSON)