mlsanityData to 14 Oct 2026

I M Avatar: Implicit Morphable Head Avatars from Videos

arXiv 2112.07471 ↗First posted 14 Dec 2021cs.CVcited by 231

Not on the map. Why?

Abstract (arXiv)

Traditional 3D morphable face models (3DMMs) provide fine-grained control over expression but cannot easily capture geometric and appearance details. Neural volumetric representations approach photorealism but are hard to animate and do not generalize well to unseen expressions. To tackle this problem, we propose IMavatar (Implicit Morphable avatar), a novel method for learning implicit head avatars from monocular videos. Inspired by the fine-grained control mechanisms afforded by conventional 3DMMs, we represent the expression- and pose- related deformations via learned blendshapes and skinning fields. These attributes are pose-independent and can be used to morph the canonical geometry and texture fields given novel expression and pose parameters. We employ ray marching and iterative root-finding to locate the canonical surface intersection for each pixel. A key contribution is our novel analytical gradient formulation that enables end-to-end training of IMavatars from videos. We show quantitatively and qualitatively that our method improves geometry and covers a more complete expression space compared to state-of-the-art methods.

Key facts. Cited by 231 papers in the map (data through 14 Oct 2026). Builds on 49 of its 74 references that are in the map. First posted on arXiv 14 Dec 2021.

Builds on

49 of 74 references.

References · 49

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2016Improved Techniques for Training GANscited by 2,970
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  7. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  8. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  9. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  10. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  11. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  12. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 632
37 more
  1. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 525
  2. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 482
  3. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 471
  4. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  5. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  6. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 390
  7. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 268
  8. 2019Capture, Learning, and Synthesis of 3D Speaking Stylescited by 259
  9. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  10. 2021HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fieldscited by 214
  11. 2021Neural Head Avatars from Monocular RGB Videoscited by 213
  12. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  13. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  14. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  15. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  16. 2018Nonlinear 3D Face Morphable Modelcited by 157
  17. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
  18. 2019NASA: Neural Articulated Shape Approximationcited by 149
  19. 2020Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatarscited by 118
  20. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  21. 2021LEAP: Learning Articulated Occupancy of Peoplecited by 107
  22. 2021Neural Lumigraph Renderingcited by 105
  23. 2021Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothingcited by 102
  24. 2017Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hzcited by 91
  25. 2020AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"cited by 91
  26. 2021H3D-Net: Few-Shot High-Fidelity 3D Head Reconstructioncited by 81
  27. 2018FML: Face Model Learning from Videoscited by 78
  28. 2020Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistencycited by 78
  29. 2020Learning Compositional Radiance Fields of Dynamic Human Headscited by 78
  30. 2019Towards a complete 3D morphable model of the human headcited by 63
  31. 2021Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracingcited by 48
  32. 2021Riggable 3D Face Reconstruction via In-Network Optimizationcited by 31
  33. 2021VariTex: Variational Neural Face Texturescited by 30
  34. 2020Personalized Face Modeling for Improved Face Reconstruction and Motion Retargetingcited by 27
  35. 2020NiLBS: Neural Inverse Linear Blend Skinningcited by 16
  36. 2018High-Quality Face Capture Using Anatomical Musclescited by 8
  37. 2020JNR: Joint-based Neural Rig Representation for Compact 3D Face Modelingcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 33; 2023: 79; 2024: 48; 2025: 43; 2026: 28.

231 papers.

Sort:

By area:

  1. 2023GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussianscited by 260
  2. 2023NeRSemble: Multi-view Radiance Field Reconstruction of Human Headscited by 194
  3. 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 184
  4. 2024SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splattingcited by 172
  5. 2022Instant Volumetric Head Avatarscited by 159
  6. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 158
  7. 2022SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Videocited by 144
  8. 2022ARAH: Animatable Volume Rendering of Articulated Human SDFscited by 139
  9. 2023Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decompositioncited by 134
  10. 2022Reconstructing Personalized Semantic Facial NeRF Models From Monocular Videocited by 128
  11. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  12. 2022Realistic One-shot Mesh-based Head Avatarscited by 119
38 more
  1. 2023FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embeddingcited by 108
  2. 2022Generative Neural Articulated Radiance Fieldscited by 101
  3. 2022EventNeRF: Neural Radiance Fields from a Single Colour Event Cameracited by 92
  4. 2023MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatarcited by 89
  5. 2023SyncTalk: The Devil is in the Synchronization for Talking Head Synthesiscited by 86
  6. 2023AvatarReX: Real-time Expressive Full-body Avatarscited by 84
  7. 2023Efficient Emotional Adaptation for Audio-Driven Talking-Head Generationcited by 81
  8. 2022KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypointscited by 77
  9. 2022Imitator: Personalized Speech-driven 3D Facial Animationcited by 74
  10. 2024GPAvatar: Generalizable and Precise Head Avatar from Image(s)cited by 74
  11. 2022AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxelscited by 73
  12. 2022Learning Neural Parametric Head Modelscited by 73
  13. 2022gDNA: Towards Generative Detailed Neural Avatarscited by 72
  14. 2023OTAvatar: One-shot Talking Face Avatar with Controllable Tri-plane Renderingcited by 67
  15. 2023X-Avatar: Expressive Human Avatarscited by 66
  16. 2024EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatarscited by 66
  17. 2024Advances in 3D Generation: A Surveycited by 63
  18. 2023HeadGaS: Real-Time Animatable Head Avatars via 3D Gaussian Splattingcited by 60
  19. 2022Fast-SNARF: A Fast Deformer for Articulated Neural Fieldscited by 55
  20. 2023Learning Locally Editable Virtual Humanscited by 54
  21. 2023NOFA: NeRF-based One-shot Facial Avatar Reconstructioncited by 53
  22. 2025Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatarscited by 53
  23. 2023StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Videocited by 52
  24. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  25. 2023LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatarcited by 50
  26. 2023HeadSculpt: Crafting 3D Head Avatars with Textcited by 49
  27. 2022PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequencecited by 46
  28. 2023GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivationcited by 46
  29. 2023RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatarscited by 45
  30. 2023Learning Personalized High Quality Volumetric Head Avatars from Monocular RGB Videoscited by 44
  31. 2023Learning Disentangled Avatars with Hybrid 3D Representationscited by 44
  32. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  33. 2023Neural Haircut: Prior-Guided Strand-Based Hair Reconstructioncited by 43
  34. 2024URAvatar: Universal Relightable Gaussian Codec Avatarscited by 43
  35. 2023DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatarscited by 42
  36. 2023Text-Guided Generation and Editing of Compositional 3D Avatarscited by 39
  37. 2023FLARE: Fast Learning of Animatable and Relightable Mesh Avatarscited by 39
  38. 2023High-Fidelity Clothed Avatar Reconstruction from a Single Imagecited by 38

50 most cited of 231 All 231 (JSON)