mlsanityData to 7 Oct 2026

Portrait4D-v2: Pseudo Multi-View Data Creates Better 4D Head Synthesizer

arXiv 2403.13570 ↗First posted 20 Mar 2024cs.CVcited by 64

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a novel learning approach for feed-forward one-shot 4D head avatar synthesis. Different from existing methods that often learn from reconstructing monocular videos guided by 3DMM, we employ pseudo multi-view videos to learn a 4D head synthesizer in a data-driven manner, avoiding reliance on inaccurate 3DMM reconstruction that could be detrimental to the synthesis performance. The key idea is to first learn a 3D head synthesizer using synthetic multi-view images to convert monocular real videos into multi-view ones, and then utilize the pseudo multi-view videos to learn a 4D head synthesizer via cross-view self-reenactment. By leveraging a simple vision transformer backbone with motion-aware cross-attentions, our method exhibits superior performance compared to previous methods in terms of reconstruction fidelity, geometry consistency, and motion control accuracy. We hope our method offers novel insights into integrating 3D priors with 2D supervisions for improved 4D head avatar creation.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 20 Mar 2024.

Builds on

46 of 59 references.

References · 46

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  8. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  9. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  10. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  11. 2020First Order Motion Model for Image Animationcited by 698
  12. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
34 more
  1. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 475
  2. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  3. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  4. 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 347
  5. 2021Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representationcited by 288
  6. 2019Few-Shot Adversarial Learning of Realistic Neural Talking Head Modelscited by 284
  7. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  8. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 254
  9. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 246
  10. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  11. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 202
  12. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  13. 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 174
  14. 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
  15. 2022MegaPortraits: One-shot Megapixel Neural Head Avatarscited by 131
  16. 2022Realistic One-shot Mesh-based Head Avatarscited by 119
  17. 2020Neural Head Reenactment with Latent Pose Descriptorscited by 106
  18. 2023CelebV-Text: A Large-Scale Facial Text-Video Datasetcited by 103
  19. 2022Generative Neural Articulated Radiance Fieldscited by 101
  20. 2024Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesiscited by 97
  21. 2021MoFaNeRF: Morphable Facial Neural Radiance Fieldcited by 94
  22. 2022Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head Synthesiscited by 93
  23. 2024GPAvatar: Generalizable and Precise Head Avatar from Image(s)cited by 74
  24. 2022MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptationcited by 69
  25. 2023One-Shot High-Fidelity Talking-Head Synthesis with Deformable Neural Radiance Fieldcited by 68
  26. 2023OTAvatar: One-shot Talking Face Avatar with Controllable Tri-plane Renderingcited by 67
  27. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  28. 20223D GAN Inversion for Controllable Portrait Image Animationcited by 53
  29. 2023NOFA: NeRF-based One-shot Facial Avatar Reconstructioncited by 53
  30. 2019Disentangled Representation Learning for 3D Face Shapecited by 48
  31. 2023GAIA: Zero-shot Talking Avatar Generationcited by 43
  32. 2020Deep 3D Portrait from a Single Imagecited by 38
  33. 2023VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactmentcited by 33
  34. 2023Generalizable One-shot Neural Head Avatarcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 30; 2026: 25.

64 papers.

Sort:

By area:

  1. 2024Generalizable and Animatable Gaussian Head Avatarcited by 95
  2. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  3. 2025Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatarscited by 52
  4. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  5. 2025LAM: Large Avatar Model for One-shot Animatable Gaussian Headcited by 43
  6. 2024GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusioncited by 36
  7. 2025ARTalk: Speech-Driven 3D Head Animation via Autoregressive Modelcited by 28
  8. 2025GUAVA: Generalizable Upper Body 3D Gaussian Avatarcited by 22
  9. 2024VOODOO XP: Expressive One-Shot Head Reenactment for VR Telepresencecited by 19
  10. 2024Joker: Conditional 3D Head Synthesis with Extreme Facial Expressionscited by 17
  11. 2025MVP4D: Multi-View Portrait Video Diffusion for Animatable 4D Avatarscited by 17
  12. 2025AvatarArtist: Open-Domain 4D Avatarizationcited by 15
38 more
  1. 2025MVPortrait: Text-Guided Motion and Emotion Control for Multi-view Vivid Portrait Animationcited by 14
  2. 2025DiffPortrait360: Consistent Portrait Diffusion for 360 View Synthesiscited by 12
  3. 2025SEGA: Drivable 3D Gaussian Head Avatar from a Single Imagecited by 11
  4. 2025FlexAvatar: Learning Complete 3D Head Avatars with Partial Supervisioncited by 10
  5. 2025LMME3DHF: Benchmarking and Evaluating Multimodal 3D Human Face Generation with LMMscited by 7
  6. 2025PanoLAM: Large Avatar Model for Gaussian Full-Head Synthesis from One-shot Unposed Imagecited by 7
  7. 2025PercHead: Perceptual Head Model for Single-Image 3D Head Reconstruction & Editingcited by 6
  8. 2026UIKA: Fast Universal Head Avatar from Pose-Free Imagescited by 5
  9. 2024Single Image, Any Face: Generalisable 3D Face Generationcited by 4
  10. 2025Controlling Avatar Diffusion with Learnable Gaussian Embeddingcited by 3
  11. 2025SpinMeRound: Consistent Multi-View Identity Generation Using Diffusion Modelscited by 3
  12. 2025Bringing Your Portrait to 3D Presencecited by 3
  13. 2025AGORA: Adversarial Generation Of Real-time Animatable 3D Gaussian Head Avatarscited by 3
  14. 2025VASA-3D: Lifelike Audio-Driven Gaussian Head Avatars from a Single Imagecited by 3
  15. 2025Snapmoji: Instant Generation of Animatable Dual-Stylized Avatarscited by 2
  16. 2025Total-Editing: Head Avatar with Editable Appearance, Motion, and Lightingcited by 2
  17. 2025RigAnyFace: Scaling Neural Facial Mesh Auto-Rigging with Unlabeled Datacited by 2
  18. 2025GaussianHeadTalk: Wobble-Free 3D Talking Heads with Audio Driven Gaussian Splattingcited by 2
  19. 2025FlexAvatar: Flexible Large Reconstruction Model for Animatable Gaussian Head Avatars with Detailed Deformationcited by 2
  20. 2024Towards High-Fidelity 3D Portrait Generation with Rich Details by Cross-View Prior-Aware Diffusioncited by 1
  21. 2025IM-Portrait: Learning 3D-aware Video Diffusion for Photorealistic Talking Heads from Monocular Videoscited by 1
  22. 2025Test-Time Augmentation for Pose-invariant Face Recognitioncited by 1
  23. 2026OMG-Avatar: One-shot Multi-LOD Gaussian Head Avatarcited by 1
  24. 2026AvatarPointillist: AutoRegressive 4D Gaussian Avatarizationcited by 1
  25. 2026MeshLAM: Feed-Forward One-Shot Animatable Textured Mesh Avatar Reconstructioncited by 1
  26. 2026Multi-view Consistent 3D Gaussian Head Avatars 'without' Multi-view Generationcited by 1
  27. 2024TimeWalker: Personalized Neural Space for Lifelong Head Avatarscited by 0
  28. 2025Low-Rank Head Avatar Personalization with Registerscited by 0
  29. 2025GRMM: Real-Time High-Fidelity Gaussian Morphable Head Model with Learned Residualscited by 0
  30. 2025Reconstruction and Reenactment Separated Method for Realistic Gaussian Headcited by 0
  31. 2025STARCaster: Spatio-Temporal AutoRegressive Video Diffusion for Identity- and View-Aware Talking Portraitscited by 0
  32. 2025Instant Expressive Gaussian Head Avatars at Over 100 FPScited by 0
  33. 2026One-Shot Feed-Forward 360circ Animatable Avatar via Inpainted UV-Space Gaussian Modelingcited by 0
  34. 2026OMEGA-Avatar: One-shot Modeling of 360deg Gaussian Avatarscited by 0
  35. 2026ExpPortrait: Expressive Portrait Generation via Personalized Representationcited by 0
  36. 2026GeoDiff4D: Geometry-Aware Diffusion for 4D Head Avatar Reconstructioncited by 0
  37. 2026LiftAvatar: Kinematic-Space Completion for Expression-Controlled 3D Gaussian Avatar Animationcited by 0
  38. 2026PerformRecast: Expression and Head Pose Disentanglement for Portrait Video Editingcited by 0

50 most cited of 64 All 64 (JSON)