# PV3D: A 3D Generative Model for Portrait Video Generation

arXiv: https://arxiv.org/abs/2212.06384 · HTML: https://mlsanity.com/paper/2212.06384/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 57 references that are in the map.
- First posted on arXiv 13 Dec 2022.

## Builds on (most cited first, 30 of 49)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Generating Videos with Scene Dynamics](https://mlsanity.com/paper/1609.02612/) (2016)
- [StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2](https://mlsanity.com/paper/2112.14683/) (2021)
- [StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation](https://mlsanity.com/paper/2112.11427/) (2021)
- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [NeRF-Editing: Geometry Editing of Neural Radiance Fields](https://mlsanity.com/paper/2205.04978/) (2022)

## Built on by (most cited first, 23 of 23)

- [MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model](https://mlsanity.com/paper/2311.16498/) (2023)
- [VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control](https://mlsanity.com/paper/2407.12781/) (2024)
- [AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers](https://mlsanity.com/paper/2411.18673/) (2024)
- [Generalizable and Animatable Gaussian Head Avatar](https://mlsanity.com/paper/2410.07971/) (2024)
- [CC3D: Layout-Conditioned Generation of Compositional 3D Scenes](https://mlsanity.com/paper/2303.12074/) (2023)
- [CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models](https://mlsanity.com/paper/2412.12093/) (2024)
- [LAM: Large Avatar Model for One-shot Animatable Gaussian Head](https://mlsanity.com/paper/2502.17796/) (2025)
- [Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation](https://mlsanity.com/paper/2509.19296/) (2025)
- [VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactment](https://mlsanity.com/paper/2312.04651/) (2023)
- [GETAvatar: Generative Textured Meshes for Animatable Human Avatars](https://mlsanity.com/paper/2310.02714/) (2023)
- [Generalizable One-shot Neural Head Avatar](https://mlsanity.com/paper/2306.08768/) (2023)
- [Real-time 3D-aware Portrait Video Relighting](https://mlsanity.com/paper/2410.18355/) (2024)
- [VOODOO XP: Expressive One-Shot Head Reenactment for VR Telepresence](https://mlsanity.com/paper/2405.16204/) (2024)
- [XAGen: 3D Expressive Human Avatars Generation](https://mlsanity.com/paper/2311.13574/) (2023)
- [ShowRoom3D: Text to High-Quality 3D Room Generation Using 3D Priors](https://mlsanity.com/paper/2312.13324/) (2023)
- [OrthoPlanes: A Novel Representation for Better 3D-Awareness of GANs](https://mlsanity.com/paper/2309.15830/) (2023)
- [Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functions](https://mlsanity.com/paper/2507.02900/) (2025)
- [Coherent 3D Portrait Video Reconstruction via Triplane Fusion](https://mlsanity.com/paper/2405.00794/) (2024)
- [Total-Editing: Head Avatar with Editable Appearance, Motion, and Lighting](https://mlsanity.com/paper/2505.20582/) (2025)
- [MeshLAM: Feed-Forward One-Shot Animatable Textured Mesh Avatar Reconstruction](https://mlsanity.com/paper/2604.22865/) (2026)
- [FAAC: Facial Animation Generation with Anchor Frame and Conditional Control for Superior Fidelity and Editability](https://mlsanity.com/paper/2312.03775/) (2023)
- [Coherent3D: Coherent 3D Portrait Video Reconstruction via Triplane Fusion](https://mlsanity.com/paper/2412.08684/) (2024)
- [Reconstruction and Reenactment Separated Method for Realistic Gaussian Head](https://mlsanity.com/paper/2509.05582/) (2025)

## Abstract (arXiv)

Recent advances in generative adversarial networks (GANs) have demonstrated the capabilities of generating stunning photo-realistic portrait images. While some prior works have applied such image GANs to unconditional 2D portrait video generation and static 3D portrait synthesis, there are few works successfully extending GANs for generating 3D-aware portrait videos. In this work, we propose PV3D, the first generative framework that can synthesize multi-view consistent portrait videos. Specifically, our method extends the recent static 3D-aware image GAN to the video domain by generalizing the 3D implicit neural representation to model the spatio-temporal space. To introduce motion dynamics to the generation process, we develop a motion generator by stacking multiple motion layers to generate motion features via modulated convolution. To alleviate motion ambiguities caused by camera/human motions, we propose a simple yet effective camera condition strategy for PV3D, enabling both temporal and multi-view consistent video generation. Moreover, PV3D introduces two discriminators for regularizing the spatial and temporal domains to ensure the plausibility of the generated portrait videos. These elaborated designs enable PV3D to generate 3D-aware motion-plausible portrait videos with high-quality appearance and geometry, significantly outperforming prior works. As a result, PV3D is able to support many downstream applications such as animating static portraits and view-consistent video motion editing. Code and models are released at https://showlab.github.io/pv3d.