mlsanityData to 14 Oct 2026

DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars

arXiv 2311.18635 ↗First posted 30 Nov 2023cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

DiffusionAvatars synthesizes a high-fidelity 3D head avatar of a person, offering intuitive control over both pose and expression. We propose a diffusion-based neural renderer that leverages generic 2D priors to produce compelling images of faces. For coarse guidance of the expression and head pose, we render a neural parametric head model (NPHM) from the target viewpoint, which acts as a proxy geometry of the person. Additionally, to enhance the modeling of intricate facial expressions, we condition DiffusionAvatars directly on the expression codes obtained from NPHM via cross-attention. Finally, to synthesize consistent surface details across different viewpoints and expressions, we rig learnable spatial features to the head's surface via TriPlane lookup in NPHM's canonical space. We train DiffusionAvatars on RGB videos and corresponding fitted NPHM meshes of a person and test the obtained avatars in both self-reenactment and animation scenarios. Our experiments demonstrate that DiffusionAvatars generates temporally consistent and visually appealing videos for novel poses and expressions of a person, outperforming existing approaches.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Builds on 72 of its 100 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

72 of 100 references.

References · 72

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  5. 2022Generative Adversarial Networkscited by 5,627
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  9. 2022Video Diffusion Modelscited by 2,305
  10. 2023Consistency Modelscited by 1,822
  11. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  12. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
38 more
  1. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  2. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  3. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  4. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  5. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  6. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  7. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  8. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  9. 2022On Distillation of Guided Diffusion Modelscited by 725
  10. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 544
  11. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 477
  12. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  13. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 444
  14. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
  15. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
  16. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  17. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 310
  18. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  19. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  20. 2018Deep Video Portraitscited by 267
  21. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  22. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 231
  23. 2020Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learningcited by 220
  24. 2021Neural Head Avatars from Monocular RGB Videoscited by 213
  25. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusioncited by 204
  26. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  27. 2021General Facial Representation Learning in a Visual-Linguistic Mannercited by 195
  28. 2023NeRSemble: Multi-view Radiance Field Reconstruction of Human Headscited by 194
  29. 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 184
  30. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 178
  31. 2020Real-Time High-Resolution Background Mattingcited by 173
  32. 2022Instant Volumetric Head Avatarscited by 159
  33. 2023DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Modelscited by 154
  34. 2023Diffused Heads: Diffusion Models Beat GANs on Talking-Face Generationcited by 142
  35. 2022RigNeRF: Fully Controllable Neural 3D Portraitscited by 126
  36. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  37. 2018FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Facescited by 119
  38. 2022Realistic One-shot Mesh-based Head Avatarscited by 119

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 18; 2025: 15; 2026: 8.

42 papers.

Sort:

By area:

  1. 2024Arc2Face: A Foundation Model for ID-Consistent Human Facescited by 101
  2. 2025Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatarscited by 53
  3. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  4. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  5. 2024FLOAT: Generative Motion Latent Flow Matching for Audio-driven Talking Portraitcited by 40
  6. 2023MonoNPHM: Dynamic Head Reconstruction from Monocular Videoscited by 35
  7. 2024HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splattingcited by 35
  8. 2024Mani-GS: Gaussian Splatting Manipulation with Triangular Meshcited by 22
  9. 2024Gaussian Eigen Models for Human Headscited by 21
  10. 2025Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstructioncited by 20
  11. 2024PSAvatar: A Point-based Shape Model for Real-Time Head Avatar Animation with 3D Gaussian Splattingcited by 17
  12. 2024PERSE: Personalized 3D Generative Avatars from A Single Portraitcited by 15
30 more
  1. 2025Synthetic Prior for Few-Shot Drivable Head Avatar Inversioncited by 14
  2. 2025Zero-1-to-A: Zero-Shot One Image to Animatable Head Avatars Using Video Diffusioncited by 9
  3. 2024DNF: Unconditional 4D Generation with Dictionary-based Neural Fieldscited by 8
  4. 2024ConsistentAvatar: Learning to Diffuse Fully Consistent Talking Head Avatar with Temporal Guidancecited by 7
  5. 2025GHOST 2.0: generative high-fidelity one shot transfer of headscited by 5
  6. 2025MagicPortrait: Temporally Consistent Face Reenactment with 3D Geometric Guidancecited by 4
  7. 2025DeX-Portrait: Disentangled and Expressive Portrait Animation via Explicit and Latent Motion Representationscited by 4
  8. 2025Controlling Avatar Diffusion with Learnable Gaussian Embeddingcited by 3
  9. 2024Revealing Directions for Text-guided 3D Face Editingcited by 2
  10. 2025TalkingPose: Efficient Face and Gesture Animation with Feedback-guided Diffusion Modelcited by 2
  11. 2025TexAvatars : Hybrid Texel-3D Representations for Stable Rigging of Photorealistic Gaussian Head Avatarscited by 2
  12. 2024Towards High-Fidelity 3D Portrait Generation with Rich Details by Cross-View Prior-Aware Diffusioncited by 1
  13. 2025LightHeadEd: Relightable & Editable Head Avatars from a Smartphonecited by 1
  14. 2025FactorPortrait: Controllable Portrait Animation via Disentangled Expression, Pose, and Viewpointcited by 1
  15. 2025Pix2NPHM: Learning to Regress NPHM Reconstructions From a Single Imagecited by 1
  16. 2026Spherical Geometry Diffusion: Generating High-quality 3D Face Geometry via Sphere-anchored Representationscited by 1
  17. 2026Multi-view Consistent 3D Gaussian Head Avatars 'without' Multi-view Generationcited by 1
  18. 2024Multimodal Conditional 3D Face Geometry Generationcited by 0
  19. 2024PainDiffusion: Learning to Express Paincited by 0
  20. 2024Pomo3D: 3D-Aware Portrait Accessorizing and Morecited by 0
  21. 2024A multidimensional measurement of photorealistic avatar quality of experiencecited by 0
  22. 2024TimeWalker: Personalized Neural Space for Lifelong Head Avatarscited by 0
  23. 2025Low-Rank Head Avatar Personalization with Registerscited by 0
  24. 2025StyleYourSmile: Cross-Domain Face Retargeting Without Paired Multi-Style Datacited by 0
  25. 2026LiftAvatar: Kinematic-Space Completion for Expression-Controlled 3D Gaussian Avatar Animationcited by 0
  26. 20263DRealHead: Few-Shot Detailed Head Avatarcited by 0
  27. 2026Loki: Representation over Architecture for Diffusion-Based Portrait Animationcited by 0
  28. 2026Self-Learning Expression Deformations for Data-Efficient Gaussian Avatarscited by 0
  29. 2026Astrolabe: Spherical-Map Guidance Across Diffusion Pipelines for Full-Body Capture from Unconstrained Imagescited by 0
  30. 2026Multi-View Face and Gesture Animation with Dynamic Gaussianscited by 0