mlsanityData to 13 Oct 2026

Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Data

arXiv 2311.18729 ↗First posted 30 Nov 2023cs.CVcited by 55

Not on the map. Why?

Abstract (arXiv)

Existing one-shot 4D head synthesis methods usually learn from monocular videos with the aid of 3DMM reconstruction, yet the latter is evenly challenging which restricts them from reasonable 4D head synthesis. We present a method to learn one-shot 4D head synthesis via large-scale synthetic data. The key is to first learn a part-wise 4D generative model from monocular images via adversarial learning, to synthesize multi-view images of diverse identities and full motions as training data; then leverage a transformer-based animatable triplane reconstructor to learn 4D head reconstruction using the synthetic data. A novel learning strategy is enforced to enhance the generalizability to real images by disentangling the learning process of 3D reconstruction and reenactment. Experiments demonstrate our superiority over the prior art.

Key facts. Cited by 55 papers in the map (data through 13 Oct 2026). Builds on 77 of its 99 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

77 of 99 references.

References · 77

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,082
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,000
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,585
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,930
  7. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,706
  8. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  9. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,150
  10. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,097
  11. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  12. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
38 more
  1. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  2. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  3. 2020First Order Motion Model for Image Animationcited by 702
  4. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  5. 2018Which Training Methods for GANs do actually Converge?cited by 676
  6. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  7. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
  8. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  9. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  10. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 476
  11. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 442
  12. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  13. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  14. 2022On the detection of synthetic images generated by diffusion modelscited by 283
  15. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 254
  16. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  17. 2020Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learningcited by 220
  18. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  19. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 202
  20. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  21. 2018X2Face: A network for controlling face generation by using images, audio, and pose codescited by 189
  22. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  23. 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 175
  24. 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
  25. 2018Face Alignment in Full Pose Range: A 3D Total Solutioncited by 147
  26. 2022MegaPortraits: One-shot Megapixel Neural Head Avatarscited by 131
  27. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  28. 2022Realistic One-shot Mesh-based Head Avatarscited by 119
  29. 2016Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Networkcited by 109
  30. 2020Neural Head Reenactment with Latent Pose Descriptorscited by 106
  31. 2022Generative Neural Articulated Radiance Fieldscited by 101
  32. 2020ID-Reveal: Identity-aware DeepFake Video Detectioncited by 97
  33. 2022Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head Synthesiscited by 93
  34. 2023PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360circcited by 93
  35. 2023StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generatorcited by 89
  36. 2022GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifoldscited by 73
  37. 20223D face reconstruction with dense landmarkscited by 70
  38. 2022MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptationcited by 69

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 9; 2025: 26; 2026: 20.

55 papers.

Sort:

By area:

  1. 2024Generalizable and Animatable Gaussian Head Avatarcited by 97
  2. 2025Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatarscited by 53
  3. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  4. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  5. 2025LAM: Large Avatar Model for One-shot Animatable Gaussian Headcited by 43
  6. 2024Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesiscited by 41
  7. 2024GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusioncited by 36
  8. 2025DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidancecited by 36
  9. 2025GUAVA: Generalizable Upper Body 3D Gaussian Avatarcited by 22
  10. 2024Joker: Conditional 3D Head Synthesis with Extreme Facial Expressionscited by 17
  11. 2025AvatarArtist: Open-Domain 4D Avatarizationcited by 15
  12. 2025Synthetic Prior for Few-Shot Drivable Head Avatar Inversioncited by 14
38 more
  1. 2024MobilePortrait: Real-Time One-Shot Neural Head Avatars on Mobile Devicescited by 12
  2. 2025DiffPortrait360: Consistent Portrait Diffusion for 360 View Synthesiscited by 12
  3. 2025SEGA: Drivable 3D Gaussian Head Avatar from a Single Imagecited by 11
  4. 2025Monocular and Generalizable Gaussian Talking Head Animationcited by 10
  5. 2025FlexAvatar: Learning Complete 3D Head Avatars with Partial Supervisioncited by 10
  6. 2025Zero-1-to-A: Zero-Shot One Image to Animatable Head Avatars Using Video Diffusioncited by 9
  7. 2025SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splattingcited by 9
  8. 2025LMME3DHF: Benchmarking and Evaluating Multimodal 3D Human Face Generation with LMMscited by 7
  9. 2025FastAvatar: Instant 3D Gaussian Splatting for Faces from Single Unconstrained Posescited by 7
  10. 2025PercHead: Perceptual Head Model for Single-Image 3D Head Reconstruction & Editingcited by 6
  11. 2025FaceCraft4D: Animated 3D Facial Avatar Generation from a Single Imagecited by 5
  12. 2026UIKA: Fast Universal Head Avatar from Pose-Free Imagescited by 5
  13. 2024Single Image, Any Face: Generalisable 3D Face Generationcited by 4
  14. 2025Controlling Avatar Diffusion with Learnable Gaussian Embeddingcited by 3
  15. 2025SpinMeRound: Consistent Multi-View Identity Generation Using Diffusion Modelscited by 3
  16. 2025Bringing Your Portrait to 3D Presencecited by 3
  17. 2025AGORA: Adversarial Generation Of Real-time Animatable 3D Gaussian Head Avatarscited by 3
  18. 2025Total-Editing: Head Avatar with Editable Appearance, Motion, and Lightingcited by 2
  19. 2025RigAnyFace: Scaling Neural Facial Mesh Auto-Rigging with Unlabeled Datacited by 2
  20. 2025GaussianHeadTalk: Wobble-Free 3D Talking Heads with Audio Driven Gaussian Splattingcited by 2
  21. 2025FlexAvatar: Flexible Large Reconstruction Model for Animatable Gaussian Head Avatars with Detailed Deformationcited by 2
  22. 2024Towards High-Fidelity 3D Portrait Generation with Rich Details by Cross-View Prior-Aware Diffusioncited by 1
  23. 2026OMG-Avatar: One-shot Multi-LOD Gaussian Head Avatarcited by 1
  24. 2026Feed-forward Gaussian Registration for Head Avatar Creation and Editingcited by 1
  25. 2026AvatarPointillist: AutoRegressive 4D Gaussian Avatarizationcited by 1
  26. 2026Large-Scale High-Quality 3D Gaussian Head Reconstruction from Multi-View Capturescited by 1
  27. 2026Multi-view Consistent 3D Gaussian Head Avatars 'without' Multi-view Generationcited by 1
  28. 2026FiCA: Feed-forward instant Gaussian Codec Avatars from a Single Portrait Imagecited by 1
  29. 2025Eye-See-You: Reverse Pass-Through VR and Head Avatarscited by 0
  30. 2025Low-Rank Head Avatar Personalization with Registerscited by 0
  31. 2026One-Shot Feed-Forward 360circ Animatable Avatar via Inpainted UV-Space Gaussian Modelingcited by 0
  32. 2026PerformRecast: Expression and Head Pose Disentanglement for Portrait Video Editingcited by 0
  33. 2026FG-Portrait: 3D Flow Guided Editable Portrait Animationcited by 0
  34. 2026Giving Faces Their Feelings Back: Explicit Emotion Control for Feedforward Single-Image 3D Head Avatarscited by 0
  35. 2026SplitAvatar: One-shot Head Avatar with Autoregressive Gaussian Splittingcited by 0
  36. 2026Self-Learning Expression Deformations for Data-Efficient Gaussian Avatarscited by 0
  37. 2026SpatialAvatar-0: High-Quality 4D Head Avatar with Multi-Stage Reconstructioncited by 0
  38. 2026FFAvatar: Feed-Forward 4D Head Avatar Reconstruction from Sparse Portrait Imagescited by 0

50 most cited of 55 All 55 (JSON)