mlsanityData to 14 Oct 2026

Generative Multiplane Images: Making a 2D GAN 3D-Aware

arXiv 2207.10642 ↗First posted 21 Jul 2022cs.CV cs.AIcited by 57

Not on the map. Why?

Abstract (arXiv)

What is really needed to make an existing 2D GAN 3D-aware? To answer this question, we modify a classical GAN, i.e., StyleGANv2, as little as possible. We find that only two modifications are absolutely necessary: 1) a multiplane image style generator branch which produces a set of alpha maps conditioned on their depth; 2) a pose-conditioned discriminator. We refer to the generated output as a 'generative multiplane image' (GMPI) and emphasize that its renderings are not only high-quality but also guaranteed to be view-consistent, which makes GMPIs different from many prior works. Importantly, the number of alpha maps can be dynamically adjusted and can differ between training and inference, alleviating memory concerns and enabling fast training of GMPIs in less than half a day at a resolution of 10242. Our findings are consistent across three challenging and common high-resolution datasets, including FFHQ, AFHQv2, and MetFaces.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Builds on 54 of its 66 references that are in the map. First posted on arXiv 21 Jul 2022.

Builds on

54 of 66 references.

References · 54

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2022Generative Adversarial Networkscited by 5,627
  4. 2013Auto-Encoding Variational Bayescited by 4,053
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  6. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  8. 2017Improved Training of Wasserstein GANscited by 2,235
  9. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  10. 2018Demystifying MMD GANscited by 1,293
  11. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  12. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
38 more
  1. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  2. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  3. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  4. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  5. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  6. 2018Which Training Methods for GANs do actually Converge?cited by 679
  7. 2017Wasserstein GANcited by 639
  8. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  9. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 482
  10. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  11. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
  12. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  13. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  14. 2017MMD GAN: Towards Deeper Understanding of Moment Matching Networkcited by 254
  15. 2018cGANs with Projection Discriminatorcited by 254
  16. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  17. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 211
  18. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  19. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  20. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  21. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  22. 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
  23. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  24. 2018Improving GANs Using Optimal Transportcited by 123
  25. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  26. 2017BEGAN: Boundary Equilibrium Generative Adversarial Networkscited by 121
  27. 2019Max-Sliced Wasserstein Distance and its use for GANscited by 119
  28. 20213D-aware Image Synthesis via Learning Structural and Textural Representationscited by 109
  29. 2018Generative Modeling using the Sliced Wasserstein Distancecited by 108
  30. 2017PacGAN: The power of two samples in generative adversarial networkscited by 104
  31. 2021GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worldscited by 98
  32. 2021A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesiscited by 72
  33. 2020A Contrastive Learning Approach for Training Variational Autoencoder Priorscited by 66
  34. 2020Lifting 2D StyleGAN for 3D-Aware Face Generationcited by 64
  35. 2017Sliced Wasserstein Distance for Learning Gaussian Mixture Modelscited by 59
  36. 2017Fisher GANcited by 22
  37. 2017McGan: Mean and Covariance Feature Matching GANcited by 21
  38. 2020Towards a Better Global Loss Landscape of GANscited by 17

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 12; 2023: 29; 2024: 11; 2025: 4; 2026: 1.

57 papers.

Sort:

By area:

  1. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  2. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
  3. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  4. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  5. 2023DiffusionRig: Learning Personalized Priors for Facial Appearance Editingcited by 75
  6. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  7. 20233D generation on ImageNetcited by 47
  8. 2024A survey of synthetic data augmentation methods in computer visioncited by 34
  9. 2022HumanGen: Generating Human Radiance Fields with Explicit Priorscited by 33
  10. 2022PV3D: A 3D Generative Model for Portrait Video Generationcited by 23
  11. 20223DFaceShop: Explicitly Controllable 3D-Aware Portrait Generationcited by 22
  12. 2023FaceLit: Neural 3D Relightable Facescited by 20
38 more
  1. 2023Local Implicit Ray Function for Generalizable Radiance Field Representationcited by 19
  2. 2024Generative Unlearning for Any Identitycited by 19
  3. 2023Unsupervised Volumetric Animationcited by 18
  4. 2023Mimic3D: Thriving 3D-Aware GANs via 3D-to-2D Imitationcited by 18
  5. 2023GANHead: Towards Generative Animatable Neural Head Avatarscited by 17
  6. 2025Compositional Generative Model of Unbounded 4D Citiescited by 17
  7. 2022A Survey on Deep Generative 3D-aware Image Synthesiscited by 16
  8. 2023VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANscited by 16
  9. 2023AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collectionscited by 16
  10. 2023Lift3D: Synthesize 3D Training Data by Lifting 2D GAN to 3D Generative Radiance Fieldcited by 15
  11. 2024Denoising Diffusion via Image-Based Renderingcited by 15
  12. 2023Learning 3D-aware Image Synthesis with Unknown Pose Distributioncited by 14
  13. 2023Efficient 3D Articulated Human Generation with Layered Surface Volumescited by 13
  14. 2023ReliTalk: Relightable Talking Portrait Generation from a Single Videocited by 12
  15. 2024What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANscited by 12
  16. 2023Ray Conditioning: Trading Photo-consistency for Photo-realism in Multi-view Image Generationcited by 11
  17. 2023SinMPI: Novel View Synthesis from a Single Image with Expanded Multiplane Imagescited by 11
  18. 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Modelscited by 10
  19. 2024GSGAN: Adversarial Learning for Hierarchical Generation of 3D Gaussian Splatscited by 9
  20. 2022Controllable Radiance Fields for Dynamic Face Synthesiscited by 8
  21. 2022Learning Detailed Radiance Manifolds for High-Fidelity and 3D-Consistent Portrait Synthesis from Monocular Imagecited by 7
  22. 2023StyleGAN Salon: Multi-View Latent Optimization for Pose-Invariant Hairstyle Transfercited by 7
  23. 2023Articulated 3D Head Avatar Generation using Text-to-Image Diffusion Modelscited by 6
  24. 2023OrthoPlanes: A Novel Representation for Better 3D-Awareness of GANscited by 6
  25. 20223D-Aware Encoding for Style-based Neural Radiance Fieldscited by 5
  26. 2022Occupancy Planes for Single-view RGB-D Human Reconstructioncited by 4
  27. 20233D-CLFusion: Fast Text-to-3D Rendering with Contrastive Latent Diffusioncited by 4
  28. 2023TEGLO: High Fidelity Canonical Texture Mapping from Single-View Imagescited by 4
  29. 2023Adversarially-Guided Portrait Mattingcited by 4
  30. 2023Benchmarking and Analyzing 3D-aware Image Synthesis with a Modularized Codebasecited by 4
  31. 2023Federated Multi-View Synthesizing for Metaversecited by 4
  32. 2024CMC: Few-shot Novel View Synthesis via Cross-view Multiplane Consistencycited by 4
  33. 2023NeRF-GAN Distillation for Efficient 3D-Aware Generation with Convolutionscited by 3
  34. 2024Learning 3D-Aware GANs from Unposed Images with Template Feature Fieldcited by 3
  35. 2024One-shot Generative Domain Adaptation in 3D GANscited by 3
  36. 2023Generative Multiplane Neural Radiance for 3D-Aware Image Generationcited by 2
  37. 2023Learning Modulated Transformation in GANscited by 1
  38. 2023Self-supervised Learning for Enhancing Geometrical Modeling in 3D-Aware Generative Adversarial Networkcited by 1

50 most cited of 57 All 57 (JSON)