mlsanityData to 14 Oct 2026

VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs

arXiv 2303.15893 ↗First posted 28 Mar 2023cs.CV cs.GR cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

We introduce VIVE3D, a novel approach that extends the capabilities of image-based 3D GANs to video editing and is able to represent the input video in an identity-preserving and temporally consistent way. We propose two new building blocks. First, we introduce a novel GAN inversion technique specifically tailored to 3D GANs by jointly embedding multiple frames and optimizing for the camera parameters. Second, besides traditional semantic face edits (e.g. for age and expression), we are the first to demonstrate edits that show novel views of the head enabled by the inherent properties of 3D GANs and our optical flow-guided compositing technique to combine the head with the background video. Our experiments demonstrate that VIVE3D generates high-fidelity face edits at consistent quality from a range of camera viewpoints which are composited with the original video in a temporally and spatially consistent manner.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 56 of its 65 references that are in the map. First posted on arXiv 28 Mar 2023.

Builds on

56 of 65 references.

References · 56

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2022Generative Adversarial Networkscited by 5,627
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  4. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  5. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  6. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  7. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  8. 2017VoxCeleb: a large-scale speaker identification datasetcited by 732
  9. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  10. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 695
  11. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  12. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
38 more
  1. 2020GANSpace: Discovering Interpretable GAN Controlscited by 614
  2. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 544
  3. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 525
  4. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  5. 2020Closed-Form Factorization of Latent Semantics in GANscited by 408
  6. 2020InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANscited by 394
  7. 2021GAN Inversion: A Surveycited by 367
  8. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  9. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 349
  10. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 301
  11. 2020Unsupervised Discovery of Interpretable Directions in the GAN Latent Spacecited by 289
  12. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  13. 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 268
  14. 2019On the "steerability" of generative adversarial networkscited by 248
  15. 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
  16. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  17. 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197
  18. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  19. 2021StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generatorscited by 164
  20. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 163
  21. 2016Inverting The Generator Of A Generative Adversarial Networkcited by 140
  22. 2022VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Gridscited by 133
  23. 2022EVA3D: Compositional 3D Human Generation from 2D Image Collectionscited by 111
  24. 2019Deep Learning for Deepfakes Creation and Detection: A Surveycited by 102
  25. 2020DeepFake Detection by Analyzing Convolutional Tracescited by 81
  26. 2022Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translationcited by 73
  27. 2022Stitch it in Time: GAN-Based Facial Editing of Real Videoscited by 68
  28. 2021A Latent Transformer for Disentangled Face Editing in Images and Videoscited by 67
  29. 2022Generative Multiplane Images: Making a 2D GAN 3D-Awarecited by 57
  30. 2022Third Time's the Charm? Image and Video Editing with StyleGAN3cited by 54
  31. 2021Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigationcited by 53
  32. 20223D GAN Inversion for Controllable Portrait Image Animationcited by 53
  33. 2020Navigating the GAN Parameter Space for Semantic Image Editingcited by 49
  34. 2021StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGANcited by 47
  35. 2022InsetGAN for Full-Body Image Generationcited by 45
  36. 20223D GAN Inversion with Pose Optimizationcited by 44
  37. 2020PIE: Portrait Image Embedding for Semantic Controlcited by 41
  38. 2022Spatially-Adaptive Multilayer Selection for GAN Inversion and Editingcited by 39

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 10; 2025: 3; 2026: 1.

16 papers.

Sort:

By area:

  1. 2024Garment3DGen: 3D Garment Stylization and Texture Generationcited by 26
  2. 2023Generalizable One-shot Neural Head Avatarcited by 21
  3. 2024Real-time 3D-aware Portrait Video Relightingcited by 21
  4. 20243D Mesh Editing using Masked LRMscited by 18
  5. 2024Geometry Transfer for Stylizing Radiance Fieldscited by 16
  6. 2024Reference-Based 3D-Aware Image Editing with Triplanescited by 9
  7. 2025FreeViS: Training-free Video Stylization with Inconsistent Referencescited by 6
  8. 2024Coherent 3D Portrait Video Reconstruction via Triplane Fusioncited by 3
  9. 2024Talk3D: High-Fidelity Talking Portrait Synthesis via Personalized 3D Generative Priorcited by 2
  10. 2024CRAFT: Designing Creative and Functional 3D Objectscited by 2
  11. 2025Vectorized Video Representation with Easy Editing via Hierarchical Spatio-Temporally Consistent Proxy Embeddingcited by 2
  12. 2023Exploring Attribute Variations in Style-based GANs using Diffusion Modelscited by 1
4 more
  1. 2024Analyzing and Improving the Skin Tone Consistency and Bias in Implicit 3D Relightable Face Generatorscited by 1
  2. 2025V-Trans4Style: Visual Transition Recommendation for Video Production Style Adaptationcited by 1
  3. 2026From Blurry to Believable: Enhancing Low-quality Talking Heads with 3D Generative Priorscited by 1
  4. 2024Coherent3D: Coherent 3D Portrait Video Reconstruction via Triplane Fusioncited by 0